Jul 02 13:12:21.282104 ip-10-0-131-238 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 02 13:12:21.282116 ip-10-0-131-238 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 02 13:12:21.282125 ip-10-0-131-238 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 02 13:12:21.282430 ip-10-0-131-238 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 02 13:12:31.421248 ip-10-0-131-238 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 02 13:12:31.421269 ip-10-0-131-238 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 9e441bb95631407bb41eb3303286b2bc --
Jul 02 13:14:08.250756 ip-10-0-131-238 systemd[1]: Starting Kubernetes Kubelet...
Jul 02 13:14:08.722529 ip-10-0-131-238 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 13:14:08.722529 ip-10-0-131-238 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 02 13:14:08.722529 ip-10-0-131-238 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 13:14:08.722529 ip-10-0-131-238 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 02 13:14:08.722529 ip-10-0-131-238 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 13:14:08.725110 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.722563    2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726642    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726653    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726656    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726659    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726662    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726666    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726669    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726671    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726674    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726677    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726679    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726687    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726690    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726692    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726695    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726698    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726700    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726716    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726718    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:14:08.750613 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726721    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726724    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726726    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726729    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726732    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726735    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726738    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726740    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726743    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726746    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726749    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726752    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726754    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726757    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726759    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726761    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726764    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726768    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726770    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726773    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:14:08.766490 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726775    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726778    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726781    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726784    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726786    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726789    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726791    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726794    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726796    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726799    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726801    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726805    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726809    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726812    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726814    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726817    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726820    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726822    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726825    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:14:08.993436 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726827    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:14:08.801185 ip-10-0-131-238 systemd[1]: Started Kubernetes Kubelet.
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726830    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726832    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726836    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726838    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726841    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726843    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726846    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726848    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726860    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726876    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726881    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726884    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726894    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726897    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726900    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726902    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726905    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726909    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726911    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:14:09.011859 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726914    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726916    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726919    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726922    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726924    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726927    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726930    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726933    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726935    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726939    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726943    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726946    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726949    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726952    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726955    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726957    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726961    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726964    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726968    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:14:09.013926 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726971    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.726974    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728901    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728910    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728914    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728917    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728920    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728924    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728926    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728929    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728933    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728935    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728938    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728940    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728943    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728945    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728950    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728952    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728955    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728958    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:14:09.016236 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728960    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728963    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728967    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728970    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728973    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728975    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728978    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728980    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728983    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728986    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728988    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728991    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728994    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728996    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.728999    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729001    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729004    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729006    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729009    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729011    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:14:09.018826 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729014    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729017    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729020    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729023    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729026    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729028    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729030    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729033    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729035    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729038    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729041    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729043    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729048    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729051    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729054    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729056    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729059    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729061    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729064    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:14:09.021299 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729067    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729069    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729072    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729074    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729077    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729080    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729084    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729087    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729090    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729093    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729095    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729099    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729102    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729104    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729108    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729113    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729116    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729118    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729121    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:14:09.022990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729124    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729126    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729129    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729132    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729134    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729137    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729140    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729144    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729148    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729151    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729154    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729157    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729160    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729162    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729165    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729168    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729173    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729176    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729178    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729181    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:14:09.024137 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729184    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729187    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729189    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729588    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729593    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729596    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729598    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729601    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729604    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729609    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729612    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729615    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729618    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729620    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729623    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729626    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729628    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729631    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729635    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729638    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:14:09.025191 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729640    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729643    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729645    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729648    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729651    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729653    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729656    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729659    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729661    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729664    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729667    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729670    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729673    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729675    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729677    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729680    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729682    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729685    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729687    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:14:09.026316 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729690    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729693    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729695    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729698    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729700    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729702    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729705    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729707    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729710    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729712    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729715    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729719    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729721    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729724    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729727    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729731    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729734    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729737    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729740    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:14:09.027185 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729742    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729745    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729747    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729751    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729753    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729756    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729758    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729761    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729764    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729766    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729769    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729771    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729773    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729776    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729779    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729781    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729784    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729786    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729789    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729791    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729795    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:14:09.028276 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729797    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729800    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729802    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729807    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729810    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729812    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729815    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729818    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729820    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729824    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729826    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729829    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729831    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729834    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729838    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729841    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729844    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729846    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729849    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729852    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:14:09.029399 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729854    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729857    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.729859    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.729941    2570 flags.go:64] FLAG: --address="0.0.0.0"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.729948    2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.729955    2570 flags.go:64] FLAG: --anonymous-auth="true"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.729962    2570 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.729970    2570 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.729974    2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.729981    2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.729988    2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.729991    2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.729994    2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.729998    2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730001    2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730006    2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730009    2570 flags.go:64] FLAG: --cgroup-root=""
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730012    2570 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730015    2570 flags.go:64] FLAG: --client-ca-file=""
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730018    2570 flags.go:64] FLAG: --cloud-provider="external"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730021    2570 flags.go:64] FLAG: --cluster-dns="[]"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730025    2570 flags.go:64] FLAG: --cluster-domain=""
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730028    2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730031    2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 02 13:14:09.030428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730035    2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730039    2570 flags.go:64] FLAG: --container-log-max-files="5"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730043    2570 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730046    2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730049    2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730053    2570 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730056    2570 flags.go:64] FLAG: --contention-profiling="false"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730059    2570 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730062    2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730065    2570 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730068    2570 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730072    2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730075    2570 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730077    2570 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730080    2570 flags.go:64] FLAG: --enable-load-reader="false"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730083    2570 flags.go:64] FLAG: --enable-server="true"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730087    2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730092    2570 flags.go:64] FLAG: --event-burst="100"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730095    2570 flags.go:64] FLAG: --event-qps="50"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730098    2570 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730101    2570 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730106    2570 flags.go:64] FLAG: --eviction-hard=""
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730109    2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730113    2570 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730117    2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 02 13:14:09.031726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730120    2570 flags.go:64] FLAG: --eviction-soft=""
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730123    2570 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730126    2570 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730129    2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730131    2570 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730134    2570 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730137    2570 flags.go:64] FLAG: --fail-swap-on="true"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730140    2570 flags.go:64] FLAG: --feature-gates=""
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730143    2570 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730147    2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730150    2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730153    2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730156    2570 flags.go:64] FLAG: --healthz-port="10248"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730159    2570 flags.go:64] FLAG: --help="false"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730162    2570 flags.go:64] FLAG: --hostname-override="ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730165    2570 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730168    2570 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730171    2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730174    2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730177    2570 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730180    2570 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730183    2570 flags.go:64] FLAG: --image-service-endpoint=""
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730186    2570 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 02 13:14:09.032791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730189    2570 flags.go:64] FLAG: --kube-api-burst="100"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730192    2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730195    2570 flags.go:64] FLAG: --kube-api-qps="50"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730197    2570 flags.go:64] FLAG: --kube-reserved=""
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730200    2570 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730204    2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730207    2570 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730211    2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730214    2570 flags.go:64] FLAG: --lock-file=""
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730217    2570 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730220    2570 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730223    2570 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730228    2570 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730231    2570 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730233    2570 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730236    2570 flags.go:64] FLAG: --logging-format="text"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730239    2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730243    2570 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730245    2570 flags.go:64] FLAG: --manifest-url=""
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730248    2570 flags.go:64] FLAG: --manifest-url-header=""
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730253    2570 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730256    2570 flags.go:64] FLAG: --max-open-files="1000000"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730260    2570 flags.go:64] FLAG: --max-pods="110"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730262    2570 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730265    2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 02 13:14:09.033854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730268    2570 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730271    2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730274    2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730277    2570 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730280    2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730287    2570 flags.go:64] FLAG: --node-status-max-images="50"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730290    2570 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730293    2570 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730296    2570 flags.go:64] FLAG: --pod-cidr=""
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730300    2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730306    2570 flags.go:64] FLAG: --pod-manifest-path=""
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730308    2570 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730313    2570 flags.go:64] FLAG: --pods-per-core="0"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730316    2570 flags.go:64] FLAG: --port="10250"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730319    2570 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730323    2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08aadccc8d589408b"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730326    2570 flags.go:64] FLAG: --qos-reserved=""
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730329    2570 flags.go:64] FLAG: --read-only-port="10255"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730332    2570 flags.go:64] FLAG: --register-node="true"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730335    2570 flags.go:64] FLAG: --register-with-taints=""
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730338    2570 flags.go:64] FLAG: --registry-burst="10"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730341    2570 flags.go:64] FLAG: --registry-qps="5"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730344    2570 flags.go:64] FLAG: --reserved-cpus=""
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730346    2570 flags.go:64] FLAG: --reserved-memory=""
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730350    2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 02 13:14:09.034930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730353    2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730356    2570 flags.go:64] FLAG: --rotate-certificates="false"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730359    2570 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730362    2570 flags.go:64] FLAG: --runonce="false"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730364    2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730367    2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730370    2570 flags.go:64] FLAG: --seccomp-default="false"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730373    2570 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730376    2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730379    2570 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730396    2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730399    2570 flags.go:64] FLAG: --storage-driver-password="root"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730402    2570 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730405    2570 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730408    2570 flags.go:64] FLAG: --storage-driver-user="root"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730411    2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730414    2570 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730417    2570 flags.go:64] FLAG: --system-cgroups=""
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730420    2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730427    2570 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730429    2570 flags.go:64] FLAG: --tls-cert-file=""
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730432    2570 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730437    2570 flags.go:64] FLAG: --tls-min-version=""
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730440    2570 flags.go:64] FLAG: --tls-private-key-file=""
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730443    2570 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 02 13:14:09.036015 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730446    2570 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730449    2570 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730452    2570 flags.go:64] FLAG: --v="2"
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730456    2570 flags.go:64] FLAG: --version="false"
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730460    2570 flags.go:64] FLAG: --vmodule=""
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730464    2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.730467    2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730587    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730591    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730595    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730598    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730601    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730604    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730607    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730609    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730612    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730615    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730617    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730620    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730622    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730625    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:14:09.037081 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730627    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730630    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730633    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730635    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730638    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730643    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730647    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730649    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730654    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730657    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730659    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730662    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730667    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730671    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730673    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730676    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730679    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730681    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730684    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:14:09.039820 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730687    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730690    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730692    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730695    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730697    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730700    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730703    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730705    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730707    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730710    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730712    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730715    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730717    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730720    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730722    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730725    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730727    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730730    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730734    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730737    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:14:09.040693 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730739    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730743    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730745    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730748    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730750    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730754    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730756    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730759    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730762    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730764    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730767    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730769    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730772    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730774    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730777    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730779    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730782    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730784    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730787    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730789    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:14:09.041570 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730792    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730794    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730797    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730799    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730802    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730804    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730807    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730809    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730812    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730814    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730819    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730823    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730825    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730829    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730831    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730834    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730838    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730842    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730844    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:14:09.042877 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730847    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:14:09.043771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730850    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:14:09.043771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730852    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:14:09.043771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730855    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:14:09.043771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730858    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:14:09.043771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730861    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:14:09.043771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.730865    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:14:09.043771 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.731534    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 13:14:09.043771 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.740266    2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 02 13:14:09.043771 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.740285    2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 02 13:14:09.043771 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.740305    2570 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 02 13:14:09.043771 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.740317    2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 02 13:14:09.043771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740376    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:14:09.043771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740380    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:14:09.043771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740398    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740401    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740404    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740407    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740410    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740412    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740415    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740417    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740420    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740423    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740426    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740428    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740431    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740433    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740436    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740438    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740441    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740443    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740446    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740449    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:14:09.044621 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740451    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740456    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740459    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740462    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740466    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740469    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740472    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740477    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740480    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740483    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740485    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740488    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740490    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740493    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740495    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740498    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740500    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740503    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740506    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:14:09.045705 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740508    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740511    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740513    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740516    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740518    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740521    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740524    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740526    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740529    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740532    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740534    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740537    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740539    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740542    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740545    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740548    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740551    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740554    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740556    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740559    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740562    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:14:09.049894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740565    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740568    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740571    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740573    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740576    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740579    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740582    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740584    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740587    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740589    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740592    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740594    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740597    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740599    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740602    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740604    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740606    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740609    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740612    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:14:09.050771 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740614    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740618    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740622    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740624    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740627    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740629    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740633    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740638    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740641    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740646    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740649    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740652    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740655    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740658    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740661    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740664    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740666    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:14:09.052038 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740669    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:14:09.052915 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.740674    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 13:14:09.052915 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740793    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:14:09.052915 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740798    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:14:09.052915 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740801    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:14:09.052915 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740804    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:14:09.052915 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740807    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:14:09.052915 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740809    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:14:09.052915 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740812    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:14:09.052915 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740815    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:14:09.052915 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740817    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:14:09.052915 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740820    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:14:09.052915 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740822    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:14:09.052915 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740825    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:14:09.052915 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740827    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:14:09.052915 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740830    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740832    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740835    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740837    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740840    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740843    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740846    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740849    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740851    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740854    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740858    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740861    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740863    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740866    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740868    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740871    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740874    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740877    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740879    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740882    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:14:09.053967 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740885    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740887    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740890    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740892    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740895    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740897    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740900    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740902    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740905    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740907    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740910    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740913    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740915    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740918    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740920    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740923    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740926    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740928    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740933    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740936    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:14:09.230292 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740938    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740941    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740945    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740947    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740950    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740952    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740955    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740958    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740963    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740966    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740970    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740973    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740976    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740979    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740982    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740985    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740989    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740992    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740995    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:14:09.231370 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.740999    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741002    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741005    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741008    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741011    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741013    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741016    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741018    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741021    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741024    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741026    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741030    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741033    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741036    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741038    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741042    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741045    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741048    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741051    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741054    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:14:09.232652 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741056    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:14:09.234140 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741059    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:14:09.234140 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741061    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:14:09.234140 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741064    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:14:09.234140 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741066    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:14:09.234140 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741069    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:14:09.234140 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:08.741071    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:14:09.234140 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.741076    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 13:14:09.234140 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.741196    2570 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 02 13:14:09.234140 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.743647    2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 02 13:14:09.234140 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.745355    2570 server.go:1019] "Starting client certificate rotation"
Jul 02 13:14:09.234140 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.745458    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 02 13:14:09.234140 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.745496    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 02 13:14:09.234140 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.750841    2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 02 13:14:09.234783 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.752163    2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 02 13:14:09.234783 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.759011    2570 log.go:25] "Validated CRI v1 runtime API"
Jul 02 13:14:09.234783 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.762842    2570 log.go:25] "Validated CRI v1 image API"
Jul 02 13:14:09.234783 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.763686    2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 02 13:14:09.234783 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.766349    2570 fs.go:135] Filesystem UUIDs: map[6c4a83de-c71d-41b9-9484-0da2ee71ec3e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8f416537-c1fe-4550-82ab-d95f5a82ec16:/dev/nvme0n1p4]
Jul 02 13:14:09.234783 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.766367    2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.772487    2570 manager.go:216] Machine: {Timestamp:2026-07-02 13:14:08.770843789 +0000 UTC m=+0.393636676 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099520 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c14d377f7a89632a8d184cb2421ca SystemUUID:ec2c14d3-77f7-a896-32a8-d184cb2421ca BootID:9e441bb9-5631-407b-b41e-b3303286b2bc Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2a:51:f4:27:c5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2a:51:f4:27:c5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:57:46:e8:86:70 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.772630    2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.772751    2570 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.776364    2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.776404    2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-238.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.777966    2570 topology_manager.go:138] "Creating topology manager with none policy"
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.777975    2570 container_manager_linux.go:309] "Creating device plugin manager"
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.777983    2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.778842    2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.780116    2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.784845    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.784968    2570 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.785857    2570 kubelet.go:486] "Attempting to sync node with API server"
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.785869    2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.785881    2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.785890    2570 kubelet.go:398] "Adding apiserver pod source"
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.785901    2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 02 13:14:09.236879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.786246    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.786265    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.790817    2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.791970    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.794045    2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.794086    2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.795575    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.795599    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.795609    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.795615    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.795624    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.795634    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.795642    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.795647    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.795658    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.795664    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.795677    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.795683    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.796533    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.796550    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:08.797182    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-238.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.800286    2570 server.go:1268] "Started kubelet"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.800366    2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.800582    2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.800642    2570 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.802070    2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.802300    2570 server.go:310] "Adding debug handlers to kubelet server"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:08.808945    2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.810818    2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.810859    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:08.811020    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-238.ec2.internal\" not found"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.811597    2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.811621    2570 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.811634    2570 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.811762    2570 reconstruct.go:97] "Volume reconstruction finished"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.811771    2570 reconciler.go:29] "Reconciler: start to sync state"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.812114    2570 factory.go:153] Registering CRI-O factory
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.812132    2570 factory.go:223] Registration of the crio container factory successfully
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.812184    2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.812206    2570 factory.go:55] Registering systemd factory
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.812214    2570 factory.go:223] Registration of the systemd container factory successfully
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.812235    2570 factory.go:103] Registering Raw factory
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.812247    2570 manager.go:1196] Started watching for new ooms in manager
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.823219    2570 manager.go:349] Starting recovery of all containers
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:08.828961    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 02 13:14:09.237736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.829151    2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-238.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:08.829364    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:08.829598    2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-238.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:08.828992    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-238.ec2.internal.18be7b4755f43020  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-238.ec2.internal,UID:ip-10-0-131-238.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-238.ec2.internal,},FirstTimestamp:2026-07-02 13:14:08.800256032 +0000 UTC m=+0.423048920,LastTimestamp:2026-07-02 13:14:08.800256032 +0000 UTC m=+0.423048920,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-238.ec2.internal,}"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.834673    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zfmc6"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.840023    2570 manager.go:354] Recovery completed
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.841432    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zfmc6"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.844676    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.846833    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.846871    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.846886    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.847494    2570 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.847505    2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.847518    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.847544    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:08.848318    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-238.ec2.internal.18be7b4758bb3516  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-238.ec2.internal,UID:ip-10-0-131-238.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-238.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-238.ec2.internal,},FirstTimestamp:2026-07-02 13:14:08.846853398 +0000 UTC m=+0.469646294,LastTimestamp:2026-07-02 13:14:08.846853398 +0000 UTC m=+0.469646294,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-238.ec2.internal,}"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.849540    2570 policy_none.go:49] "None policy: Start"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.849553    2570 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.849564    2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.851490    2570 policy_none.go:47] "Start"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.852182    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.852210    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.852223    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.890205    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.891350    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.891408    2570 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.891456    2570 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:08.891498    2570 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:08.893906    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:08.911219    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-238.ec2.internal\" not found"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:08.992201    2570 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:09.011275    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-238.ec2.internal\" not found"
Jul 02 13:14:09.240095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.025968    2570 manager.go:337] "Starting Device Plugin manager"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:09.026004    2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.026015    2570 server.go:87] "Starting device plugin registration server"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.026322    2570 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.026337    2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.026793    2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.026898    2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.026910    2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:09.027241    2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:09.027303    2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-238.ec2.internal\" not found"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:09.065992    2570 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-238.ec2.internal\" not found" node="ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.127425    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.128415    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.128449    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.128463    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.128491    2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.137748    2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:09.137768    2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-238.ec2.internal\": node \"ip-10-0-131-238.ec2.internal\" not found"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:09.183021    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-238.ec2.internal\" not found"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.192992    2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-238.ec2.internal"]
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.193068    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.194298    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.194332    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.194348    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.197310    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.197489    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.197520    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.198088    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.198104    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.198123    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.198135    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.198125    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.198231    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.200400    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.200431    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.201152    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.201175    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.201185    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:14:09.241799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.214108    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f2dc82eac687745157e1911aad1aef8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal\" (UID: \"9f2dc82eac687745157e1911aad1aef8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.244207 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.214129    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f2dc82eac687745157e1911aad1aef8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal\" (UID: \"9f2dc82eac687745157e1911aad1aef8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.244207 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.214147    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/10fd38b2c4f9fdad3bb40dd53988d42d-config\") pod \"kube-apiserver-proxy-ip-10-0-131-238.ec2.internal\" (UID: \"10fd38b2c4f9fdad3bb40dd53988d42d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.244207 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:09.220901    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-238.ec2.internal\" not found" node="ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.244207 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:09.224882    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-238.ec2.internal\" not found" node="ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.421217 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:09.283317    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-238.ec2.internal\" not found"
Jul 02 13:14:09.421217 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.315024    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f2dc82eac687745157e1911aad1aef8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal\" (UID: \"9f2dc82eac687745157e1911aad1aef8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.421217 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.315049    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f2dc82eac687745157e1911aad1aef8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal\" (UID: \"9f2dc82eac687745157e1911aad1aef8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.421217 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.315068    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/10fd38b2c4f9fdad3bb40dd53988d42d-config\") pod \"kube-apiserver-proxy-ip-10-0-131-238.ec2.internal\" (UID: \"10fd38b2c4f9fdad3bb40dd53988d42d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.421217 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.315109    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f2dc82eac687745157e1911aad1aef8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal\" (UID: \"9f2dc82eac687745157e1911aad1aef8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.421217 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.315132    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f2dc82eac687745157e1911aad1aef8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal\" (UID: \"9f2dc82eac687745157e1911aad1aef8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.421217 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.315105    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/10fd38b2c4f9fdad3bb40dd53988d42d-config\") pod \"kube-apiserver-proxy-ip-10-0-131-238.ec2.internal\" (UID: \"10fd38b2c4f9fdad3bb40dd53988d42d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.421217 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:09.383446    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-238.ec2.internal\" not found"
Jul 02 13:14:09.593698 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:09.486006    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-238.ec2.internal\" not found"
Jul 02 13:14:09.593698 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.525361    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.593698 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.530939    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.593698 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:09.588869    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-238.ec2.internal\" not found"
Jul 02 13:14:09.767417 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:09.694957    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-238.ec2.internal\" not found"
Jul 02 13:14:09.767417 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.746117    2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 02 13:14:09.767417 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.746268    2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 02 13:14:09.946671 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:09.796357    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-238.ec2.internal\" not found"
Jul 02 13:14:09.946671 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.828898    2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 13:14:09.946671 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.847093    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 13:09:09 +0000 UTC" deadline="2028-03-14 12:11:31.468697042 +0000 UTC"
Jul 02 13:14:09.946671 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.847138    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14902h57m21.621568879s"
Jul 02 13:14:09.946671 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.892315    2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 13:14:09.946671 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:09.896524    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-238.ec2.internal\" not found"
Jul 02 13:14:09.946671 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.903503    2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 13:14:09.946671 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.911262    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.946671 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.923107    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 02 13:14:09.946671 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.924233    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-238.ec2.internal"
Jul 02 13:14:09.946671 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:09.931770    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 02 13:14:10.313128 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:10.308189    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f2dc82eac687745157e1911aad1aef8.slice/crio-123cc0cdeec7c8db7b223ae96d3c65b8333521827dbbd76bd6243aa623ccc5ac WatchSource:0}: Error finding container 123cc0cdeec7c8db7b223ae96d3c65b8333521827dbbd76bd6243aa623ccc5ac: Status 404 returned error can't find the container with id 123cc0cdeec7c8db7b223ae96d3c65b8333521827dbbd76bd6243aa623ccc5ac
Jul 02 13:14:10.317861 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.317534    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 13:14:10.322113 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:10.319976    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10fd38b2c4f9fdad3bb40dd53988d42d.slice/crio-037ded6f260093b8a3bd9a1a0e6f4be02c7db8a72c0d0cb5a879634d28124a46 WatchSource:0}: Error finding container 037ded6f260093b8a3bd9a1a0e6f4be02c7db8a72c0d0cb5a879634d28124a46: Status 404 returned error can't find the container with id 037ded6f260093b8a3bd9a1a0e6f4be02c7db8a72c0d0cb5a879634d28124a46
Jul 02 13:14:10.636626 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.636602    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 13:14:10.789475 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.789440    2570 apiserver.go:52] "Watching apiserver"
Jul 02 13:14:10.794040 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.794021    2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 02 13:14:10.794480 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.794440    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal","openshift-multus/multus-additional-cni-plugins-zwphv","openshift-network-diagnostics/network-check-target-kv69m","openshift-network-operator/iptables-alerter-f4rtw","kube-system/global-pull-secret-syncer-bhrhl","kube-system/konnectivity-agent-9z5jl","kube-system/kube-apiserver-proxy-ip-10-0-131-238.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w","openshift-dns/node-resolver-wfmns","openshift-multus/multus-g4qzs","openshift-multus/network-metrics-daemon-6nhcs","openshift-ovn-kubernetes/ovnkube-node-djnhg","openshift-cluster-node-tuning-operator/tuned-tt9pg","openshift-image-registry/node-ca-q9xvb"]
Jul 02 13:14:10.797431 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.797408    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.799258 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.799233    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 02 13:14:10.799364 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.799282    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 02 13:14:10.799364 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.799315    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j4x7s\""
Jul 02 13:14:10.799364 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.799240    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 02 13:14:10.799767 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.799751    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.801829 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.801761    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 02 13:14:10.802034 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.801881    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-26d5q\""
Jul 02 13:14:10.802034 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.801924    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 02 13:14:10.802183 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.802153    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 02 13:14:10.802183 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.802171    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 02 13:14:10.802283 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.802190    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 02 13:14:10.802978 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.802954    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:10.803078 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:10.803050    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:10.805354 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.805334    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.806910 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.806887    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9pjlx\""
Jul 02 13:14:10.807282 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.807249    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 02 13:14:10.807519 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.807501    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 02 13:14:10.807789 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.807768    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:10.807890 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:10.807847    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:10.810149 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.810129    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9z5jl"
Jul 02 13:14:10.811706 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.811673    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 02 13:14:10.811817 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.811710    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 02 13:14:10.812049 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.812033    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 02 13:14:10.812349 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.812202    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qq67g\""
Jul 02 13:14:10.812799 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.812781    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.814725 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.814637    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-n4nnt\""
Jul 02 13:14:10.814725 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.814672    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 02 13:14:10.815567 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.815487    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f4rtw"
Jul 02 13:14:10.817838 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.817819    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 02 13:14:10.817932 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.817919    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 02 13:14:10.818132 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.818099    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wfmns"
Jul 02 13:14:10.818593 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.818568    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 02 13:14:10.818819 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.818795    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jzvm8\""
Jul 02 13:14:10.819864 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.819806    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5nm4r\""
Jul 02 13:14:10.820200 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.820179    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 02 13:14:10.821200 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.821178    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 02 13:14:10.823863 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.823842    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-multus-conf-dir\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.823863 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.823884    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-run\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.824110 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.823965    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-run-netns\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.824110 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824014    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11cd0142-78dd-4651-b2af-2c26f1ce70e4-system-cni-dir\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.824110 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824050    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-lib-modules\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.824110 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824082    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9325a868-a924-4308-8533-25b57f13535a-etc-tuned\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.824447 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824121    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs\") pod \"network-metrics-daemon-6nhcs\" (UID: \"aa2c8da0-b138-46ef-b9eb-10f6922697be\") " pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:10.824447 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824190    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.824447 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824227    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n57fd\" (UniqueName: \"kubernetes.io/projected/b2bd9d19-8c80-4487-b0c9-cf470b482741-kube-api-access-n57fd\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.824447 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824256    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9454c557-e08b-4fbd-92fe-a2cffddf034b-cni-binary-copy\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.824447 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824364    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-var-lib-cni-bin\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.824447 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824399    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-modprobe-d\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.824889 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824457    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-var-lib-kubelet\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.824889 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824494    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-socket-dir\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.824889 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824528    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-hostroot\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.824889 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824610    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-run-multus-certs\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.824889 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824671    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-etc-kubernetes\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.824889 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824693    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 02 13:14:10.824889 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824880    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-kubernetes\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.825286 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824933    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dplbc\" (UniqueName: \"kubernetes.io/projected/9325a868-a924-4308-8533-25b57f13535a-kube-api-access-dplbc\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.825286 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.824972    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-device-dir\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.825286 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.825045    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/11cd0142-78dd-4651-b2af-2c26f1ce70e4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.825286 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.825087    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:10.825286 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.825227    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk65x\" (UniqueName: \"kubernetes.io/projected/11cd0142-78dd-4651-b2af-2c26f1ce70e4-kube-api-access-hk65x\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.825635 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:10.825293    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:10.825635 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.825296    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-run-k8s-cni-cncf-io\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.825635 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.825413    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-var-lib-kubelet\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.825635 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.825463    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-host\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.825635 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.825502    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5dlc\" (UniqueName: \"kubernetes.io/projected/aa2c8da0-b138-46ef-b9eb-10f6922697be-kube-api-access-g5dlc\") pod \"network-metrics-daemon-6nhcs\" (UID: \"aa2c8da0-b138-46ef-b9eb-10f6922697be\") " pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:10.825635 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.825575    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-etc-selinux\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.825934 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.825628    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11cd0142-78dd-4651-b2af-2c26f1ce70e4-cni-binary-copy\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.825934 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.825687    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11cd0142-78dd-4651-b2af-2c26f1ce70e4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.826071 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.825999    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-system-cni-dir\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.826071 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826032    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-cnibin\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.826071 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826058    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-os-release\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.826252 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826083    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-sysconfig\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.826252 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826107    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-systemd\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.826252 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826132    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11cd0142-78dd-4651-b2af-2c26f1ce70e4-os-release\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.826252 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826158    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9454c557-e08b-4fbd-92fe-a2cffddf034b-multus-daemon-config\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.826252 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826202    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm68d\" (UniqueName: \"kubernetes.io/projected/9454c557-e08b-4fbd-92fe-a2cffddf034b-kube-api-access-jm68d\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.826252 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826235    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-sysctl-d\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.826572 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826257    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-sys\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.826572 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826279    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9325a868-a924-4308-8533-25b57f13535a-tmp\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.826572 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826295    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-sys-fs\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.826572 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826310    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11cd0142-78dd-4651-b2af-2c26f1ce70e4-cnibin\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.826572 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826326    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmxvp\" (UniqueName: \"kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp\") pod \"network-check-target-kv69m\" (UID: \"c41c0fa4-04c6-4e1b-9af2-0635451f3f26\") " pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:10.826572 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826349    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae596d11-edc3-4966-abdb-f76a7b0f9c94-agent-certs\") pod \"konnectivity-agent-9z5jl\" (UID: \"ae596d11-edc3-4966-abdb-f76a7b0f9c94\") " pod="kube-system/konnectivity-agent-9z5jl"
Jul 02 13:14:10.826572 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826384    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-sysctl-conf\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.826572 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826426    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-registration-dir\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.826572 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826472    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/11cd0142-78dd-4651-b2af-2c26f1ce70e4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.826572 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826495    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae596d11-edc3-4966-abdb-f76a7b0f9c94-konnectivity-ca\") pod \"konnectivity-agent-9z5jl\" (UID: \"ae596d11-edc3-4966-abdb-f76a7b0f9c94\") " pod="kube-system/konnectivity-agent-9z5jl"
Jul 02 13:14:10.826572 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826537    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-multus-cni-dir\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.826572 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826557    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-multus-socket-dir-parent\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.827119 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.826585    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-var-lib-cni-multus\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.828554 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.828505    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.830353 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.830304    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 02 13:14:10.830353 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.830341    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 02 13:14:10.830353 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.830362    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 02 13:14:10.830708 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.830478    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 02 13:14:10.830788 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.830766    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rdp69\""
Jul 02 13:14:10.830942 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.830923    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 02 13:14:10.831052 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.830966    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 02 13:14:10.831752 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.831733    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-q9xvb"
Jul 02 13:14:10.833712 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.833552    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 02 13:14:10.833712 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.833580    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cbcrc\""
Jul 02 13:14:10.833712 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.833707    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 02 13:14:10.833904 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.833774    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 02 13:14:10.879561 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.879529    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bbn57"
Jul 02 13:14:10.887679 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.887560    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bbn57"
Jul 02 13:14:10.896752 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.896676    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-238.ec2.internal" event={"ID":"10fd38b2c4f9fdad3bb40dd53988d42d","Type":"ContainerStarted","Data":"037ded6f260093b8a3bd9a1a0e6f4be02c7db8a72c0d0cb5a879634d28124a46"}
Jul 02 13:14:10.898088 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.898062    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal" event={"ID":"9f2dc82eac687745157e1911aad1aef8","Type":"ContainerStarted","Data":"123cc0cdeec7c8db7b223ae96d3c65b8333521827dbbd76bd6243aa623ccc5ac"}
Jul 02 13:14:10.912999 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.912977    2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 02 13:14:10.927541 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.927514    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8cce14c3-bcec-404c-9873-9825555f2fc4-ovn-node-metrics-cert\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.927659 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.927554    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/877fea60-1134-4e22-8db9-67000f4a4692-host-slash\") pod \"iptables-alerter-f4rtw\" (UID: \"877fea60-1134-4e22-8db9-67000f4a4692\") " pod="openshift-network-operator/iptables-alerter-f4rtw"
Jul 02 13:14:10.927659 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.927596    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx7hd\" (UniqueName: \"kubernetes.io/projected/fb05ec42-ba16-4d8b-82e1-1e689b75392d-kube-api-access-mx7hd\") pod \"node-resolver-wfmns\" (UID: \"fb05ec42-ba16-4d8b-82e1-1e689b75392d\") " pod="openshift-dns/node-resolver-wfmns"
Jul 02 13:14:10.927659 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.927629    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-run\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.927659 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.927654    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-kubelet\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.927972 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.927750    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-run-systemd\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.927972 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.927824    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8cce14c3-bcec-404c-9873-9825555f2fc4-ovnkube-script-lib\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.927972 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.927831    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-run\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.927972 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.927923    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cc015ede-26b1-49e2-9812-626d373e658f-serviceca\") pod \"node-ca-q9xvb\" (UID: \"cc015ede-26b1-49e2-9812-626d373e658f\") " pod="openshift-image-registry/node-ca-q9xvb"
Jul 02 13:14:10.927972 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.927969    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11cd0142-78dd-4651-b2af-2c26f1ce70e4-system-cni-dir\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.928244 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.927999    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs\") pod \"network-metrics-daemon-6nhcs\" (UID: \"aa2c8da0-b138-46ef-b9eb-10f6922697be\") " pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:10.928244 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928064    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11cd0142-78dd-4651-b2af-2c26f1ce70e4-system-cni-dir\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.928244 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928087    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.928244 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928118    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9454c557-e08b-4fbd-92fe-a2cffddf034b-cni-binary-copy\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.928244 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928134    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-var-lib-cni-bin\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.928244 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928149    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-hostroot\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.928244 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928168    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.928244 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928168    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-run-ovn-kubernetes\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.928244 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928197    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q92jf\" (UniqueName: \"kubernetes.io/projected/cc015ede-26b1-49e2-9812-626d373e658f-kube-api-access-q92jf\") pod \"node-ca-q9xvb\" (UID: \"cc015ede-26b1-49e2-9812-626d373e658f\") " pod="openshift-image-registry/node-ca-q9xvb"
Jul 02 13:14:10.928244 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928215    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-var-lib-kubelet\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.928244 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928226    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-var-lib-cni-bin\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.928244 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928235    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-run-multus-certs\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928270    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-node-log\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928291    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-hostroot\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928302    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-cni-netd\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:10.928177    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928374    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/877fea60-1134-4e22-8db9-67000f4a4692-iptables-alerter-script\") pod \"iptables-alerter-f4rtw\" (UID: \"877fea60-1134-4e22-8db9-67000f4a4692\") " pod="openshift-network-operator/iptables-alerter-f4rtw"
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928405    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-device-dir\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928462    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk65x\" (UniqueName: \"kubernetes.io/projected/11cd0142-78dd-4651-b2af-2c26f1ce70e4-kube-api-access-hk65x\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928488    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8cce14c3-bcec-404c-9873-9825555f2fc4-env-overrides\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928512    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-host\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928542    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11cd0142-78dd-4651-b2af-2c26f1ce70e4-cni-binary-copy\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928569    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-system-cni-dir\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928593    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-cnibin\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928619    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-os-release\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928647    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9454c557-e08b-4fbd-92fe-a2cffddf034b-multus-daemon-config\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928675    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-sysconfig\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928703    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11cd0142-78dd-4651-b2af-2c26f1ce70e4-os-release\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.928884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928727    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-run-openvswitch\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928754    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928775    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-run-multus-certs\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928795    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cc015ede-26b1-49e2-9812-626d373e658f-host\") pod \"node-ca-q9xvb\" (UID: \"cc015ede-26b1-49e2-9812-626d373e658f\") " pod="openshift-image-registry/node-ca-q9xvb"
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928824    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9454c557-e08b-4fbd-92fe-a2cffddf034b-cni-binary-copy\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:10.928848    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs podName:aa2c8da0-b138-46ef-b9eb-10f6922697be nodeName:}" failed. No retries permitted until 2026-07-02 13:14:11.428805252 +0000 UTC m=+3.051598154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs") pod "network-metrics-daemon-6nhcs" (UID: "aa2c8da0-b138-46ef-b9eb-10f6922697be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928883    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9325a868-a924-4308-8533-25b57f13535a-tmp\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928901    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-sys-fs\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928922    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae596d11-edc3-4966-abdb-f76a7b0f9c94-agent-certs\") pod \"konnectivity-agent-9z5jl\" (UID: \"ae596d11-edc3-4966-abdb-f76a7b0f9c94\") " pod="kube-system/konnectivity-agent-9z5jl"
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928948    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae596d11-edc3-4966-abdb-f76a7b0f9c94-konnectivity-ca\") pod \"konnectivity-agent-9z5jl\" (UID: \"ae596d11-edc3-4966-abdb-f76a7b0f9c94\") " pod="kube-system/konnectivity-agent-9z5jl"
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.928975    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-multus-cni-dir\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929008    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-os-release\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929295    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-multus-socket-dir-parent\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929332    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-host\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929362    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-device-dir\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929377    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-var-lib-cni-multus\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929445    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11cd0142-78dd-4651-b2af-2c26f1ce70e4-os-release\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.929759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929332    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-var-lib-cni-multus\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929524    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-sysctl-conf\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929550    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-registration-dir\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929576    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-multus-conf-dir\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929614    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-slash\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929627    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11cd0142-78dd-4651-b2af-2c26f1ce70e4-cni-binary-copy\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929637    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-cni-bin\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929659    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fb05ec42-ba16-4d8b-82e1-1e689b75392d-hosts-file\") pod \"node-resolver-wfmns\" (UID: \"fb05ec42-ba16-4d8b-82e1-1e689b75392d\") " pod="openshift-dns/node-resolver-wfmns"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929697    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-sysctl-conf\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929715    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-system-cni-dir\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929687    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret\") pod \"global-pull-secret-syncer-bhrhl\" (UID: \"a8a9f309-636c-41da-a9a7-6d33ebced403\") " pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929758    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-run-netns\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929766    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-sysconfig\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929782    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-registration-dir\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929788    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-etc-openvswitch\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929831    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-multus-conf-dir\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929852    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-cnibin\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929899    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-var-lib-kubelet\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.930566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929681    2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929892    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a8a9f309-636c-41da-a9a7-6d33ebced403-dbus\") pod \"global-pull-secret-syncer-bhrhl\" (UID: \"a8a9f309-636c-41da-a9a7-6d33ebced403\") " pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929946    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9454c557-e08b-4fbd-92fe-a2cffddf034b-multus-daemon-config\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929959    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-sys-fs\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.929951    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-lib-modules\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930002    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9325a868-a924-4308-8533-25b57f13535a-etc-tuned\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930069    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-lib-modules\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930090    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-multus-socket-dir-parent\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930125    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n57fd\" (UniqueName: \"kubernetes.io/projected/b2bd9d19-8c80-4487-b0c9-cf470b482741-kube-api-access-n57fd\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930153    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-modprobe-d\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930169    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-multus-cni-dir\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930179    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-socket-dir\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930230    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-run-netns\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930262    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-etc-kubernetes\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930273    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae596d11-edc3-4966-abdb-f76a7b0f9c94-konnectivity-ca\") pod \"konnectivity-agent-9z5jl\" (UID: \"ae596d11-edc3-4966-abdb-f76a7b0f9c94\") " pod="kube-system/konnectivity-agent-9z5jl"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930290    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-var-lib-openvswitch\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930318    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-log-socket\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.931334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930368    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-etc-kubernetes\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930373    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-socket-dir\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930476    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-modprobe-d\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930506    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-kubernetes\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930548    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dplbc\" (UniqueName: \"kubernetes.io/projected/9325a868-a924-4308-8533-25b57f13535a-kube-api-access-dplbc\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930595    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/11cd0142-78dd-4651-b2af-2c26f1ce70e4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930635    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-run-k8s-cni-cncf-io\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930775    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-var-lib-kubelet\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930831    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5dlc\" (UniqueName: \"kubernetes.io/projected/aa2c8da0-b138-46ef-b9eb-10f6922697be-kube-api-access-g5dlc\") pod \"network-metrics-daemon-6nhcs\" (UID: \"aa2c8da0-b138-46ef-b9eb-10f6922697be\") " pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930832    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-run-k8s-cni-cncf-io\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930870    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-etc-selinux\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930899    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9454c557-e08b-4fbd-92fe-a2cffddf034b-host-var-lib-kubelet\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930939    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11cd0142-78dd-4651-b2af-2c26f1ce70e4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930960    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2bd9d19-8c80-4487-b0c9-cf470b482741-etc-selinux\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.930965    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm68d\" (UniqueName: \"kubernetes.io/projected/9454c557-e08b-4fbd-92fe-a2cffddf034b-kube-api-access-jm68d\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.931045    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-kubernetes\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.931214    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11cd0142-78dd-4651-b2af-2c26f1ce70e4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.932208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.931237    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/11cd0142-78dd-4651-b2af-2c26f1ce70e4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.933069 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.932277    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-systemd-units\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.933069 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.932534    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-run-ovn\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.933069 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.932602    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a8a9f309-636c-41da-a9a7-6d33ebced403-kubelet-config\") pod \"global-pull-secret-syncer-bhrhl\" (UID: \"a8a9f309-636c-41da-a9a7-6d33ebced403\") " pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:10.933069 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.932660    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-systemd\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.933069 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.932702    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb05ec42-ba16-4d8b-82e1-1e689b75392d-tmp-dir\") pod \"node-resolver-wfmns\" (UID: \"fb05ec42-ba16-4d8b-82e1-1e689b75392d\") " pod="openshift-dns/node-resolver-wfmns"
Jul 02 13:14:10.933069 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.932754    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-sysctl-d\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.933069 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.932799    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-sys\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.933069 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.932841    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11cd0142-78dd-4651-b2af-2c26f1ce70e4-cnibin\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.933069 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.932875    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmxvp\" (UniqueName: \"kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp\") pod \"network-check-target-kv69m\" (UID: \"c41c0fa4-04c6-4e1b-9af2-0635451f3f26\") " pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:10.933069 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.932906    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-run-netns\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.933069 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.932937    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8cce14c3-bcec-404c-9873-9825555f2fc4-ovnkube-config\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.933069 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.932993    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lppgr\" (UniqueName: \"kubernetes.io/projected/8cce14c3-bcec-404c-9873-9825555f2fc4-kube-api-access-lppgr\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:10.933069 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.933049    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7kzw\" (UniqueName: \"kubernetes.io/projected/877fea60-1134-4e22-8db9-67000f4a4692-kube-api-access-t7kzw\") pod \"iptables-alerter-f4rtw\" (UID: \"877fea60-1134-4e22-8db9-67000f4a4692\") " pod="openshift-network-operator/iptables-alerter-f4rtw"
Jul 02 13:14:10.933669 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.933095    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/11cd0142-78dd-4651-b2af-2c26f1ce70e4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.933669 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.933115    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-sysctl-d\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.933669 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.933556    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-etc-systemd\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.933669 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.933625    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9325a868-a924-4308-8533-25b57f13535a-sys\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.933874 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.933698    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11cd0142-78dd-4651-b2af-2c26f1ce70e4-cnibin\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.933874 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.933779    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/11cd0142-78dd-4651-b2af-2c26f1ce70e4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.933998 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.933971    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9325a868-a924-4308-8533-25b57f13535a-etc-tuned\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.936641 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.935795    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9325a868-a924-4308-8533-25b57f13535a-tmp\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.937675 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.937649    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae596d11-edc3-4966-abdb-f76a7b0f9c94-agent-certs\") pod \"konnectivity-agent-9z5jl\" (UID: \"ae596d11-edc3-4966-abdb-f76a7b0f9c94\") " pod="kube-system/konnectivity-agent-9z5jl"
Jul 02 13:14:10.938672 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:10.938650    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:14:10.938798 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:10.938675    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:14:10.938798 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:10.938691    2570 projected.go:196] Error preparing data for projected volume kube-api-access-pmxvp for pod openshift-network-diagnostics/network-check-target-kv69m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:14:10.938798 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:10.938749    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp podName:c41c0fa4-04c6-4e1b-9af2-0635451f3f26 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:11.438731063 +0000 UTC m=+3.061523971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pmxvp" (UniqueName: "kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp") pod "network-check-target-kv69m" (UID: "c41c0fa4-04c6-4e1b-9af2-0635451f3f26") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:14:10.940696 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.940636    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk65x\" (UniqueName: \"kubernetes.io/projected/11cd0142-78dd-4651-b2af-2c26f1ce70e4-kube-api-access-hk65x\") pod \"multus-additional-cni-plugins-zwphv\" (UID: \"11cd0142-78dd-4651-b2af-2c26f1ce70e4\") " pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:10.941188 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.941165    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm68d\" (UniqueName: \"kubernetes.io/projected/9454c557-e08b-4fbd-92fe-a2cffddf034b-kube-api-access-jm68d\") pod \"multus-g4qzs\" (UID: \"9454c557-e08b-4fbd-92fe-a2cffddf034b\") " pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:10.941363 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.941340    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dplbc\" (UniqueName: \"kubernetes.io/projected/9325a868-a924-4308-8533-25b57f13535a-kube-api-access-dplbc\") pod \"tuned-tt9pg\" (UID: \"9325a868-a924-4308-8533-25b57f13535a\") " pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:10.941843 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.941819    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n57fd\" (UniqueName: \"kubernetes.io/projected/b2bd9d19-8c80-4487-b0c9-cf470b482741-kube-api-access-n57fd\") pod \"aws-ebs-csi-driver-node-pdq9w\" (UID: \"b2bd9d19-8c80-4487-b0c9-cf470b482741\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:10.941918 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:10.941881    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5dlc\" (UniqueName: \"kubernetes.io/projected/aa2c8da0-b138-46ef-b9eb-10f6922697be-kube-api-access-g5dlc\") pod \"network-metrics-daemon-6nhcs\" (UID: \"aa2c8da0-b138-46ef-b9eb-10f6922697be\") " pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:11.033858 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033591    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-run-openvswitch\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.033858 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033640    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.033858 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033680    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cc015ede-26b1-49e2-9812-626d373e658f-host\") pod \"node-ca-q9xvb\" (UID: \"cc015ede-26b1-49e2-9812-626d373e658f\") " pod="openshift-image-registry/node-ca-q9xvb"
Jul 02 13:14:11.033858 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033719    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-slash\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.033858 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033746    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-cni-bin\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.033858 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033753    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cc015ede-26b1-49e2-9812-626d373e658f-host\") pod \"node-ca-q9xvb\" (UID: \"cc015ede-26b1-49e2-9812-626d373e658f\") " pod="openshift-image-registry/node-ca-q9xvb"
Jul 02 13:14:11.033858 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033772    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fb05ec42-ba16-4d8b-82e1-1e689b75392d-hosts-file\") pod \"node-resolver-wfmns\" (UID: \"fb05ec42-ba16-4d8b-82e1-1e689b75392d\") " pod="openshift-dns/node-resolver-wfmns"
Jul 02 13:14:11.033858 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033717    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-run-openvswitch\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.033858 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033719    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.033858 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033796    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-slash\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.033858 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033800    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret\") pod \"global-pull-secret-syncer-bhrhl\" (UID: \"a8a9f309-636c-41da-a9a7-6d33ebced403\") " pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:11.033858 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033856    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fb05ec42-ba16-4d8b-82e1-1e689b75392d-hosts-file\") pod \"node-resolver-wfmns\" (UID: \"fb05ec42-ba16-4d8b-82e1-1e689b75392d\") " pod="openshift-dns/node-resolver-wfmns"
Jul 02 13:14:11.033858 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033818    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-cni-bin\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033892    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-etc-openvswitch\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033922    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a8a9f309-636c-41da-a9a7-6d33ebced403-dbus\") pod \"global-pull-secret-syncer-bhrhl\" (UID: \"a8a9f309-636c-41da-a9a7-6d33ebced403\") " pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033955    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-var-lib-openvswitch\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033988    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-log-socket\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.033996    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-etc-openvswitch\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:11.034021    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034039    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-var-lib-openvswitch\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:11.034082    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret podName:a8a9f309-636c-41da-a9a7-6d33ebced403 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:11.534060791 +0000 UTC m=+3.156853699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret") pod "global-pull-secret-syncer-bhrhl" (UID: "a8a9f309-636c-41da-a9a7-6d33ebced403") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034088    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-systemd-units\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034075    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-log-socket\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034105    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a8a9f309-636c-41da-a9a7-6d33ebced403-dbus\") pod \"global-pull-secret-syncer-bhrhl\" (UID: \"a8a9f309-636c-41da-a9a7-6d33ebced403\") " pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034021    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-systemd-units\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034164    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-run-ovn\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034194    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a8a9f309-636c-41da-a9a7-6d33ebced403-kubelet-config\") pod \"global-pull-secret-syncer-bhrhl\" (UID: \"a8a9f309-636c-41da-a9a7-6d33ebced403\") " pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034226    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb05ec42-ba16-4d8b-82e1-1e689b75392d-tmp-dir\") pod \"node-resolver-wfmns\" (UID: \"fb05ec42-ba16-4d8b-82e1-1e689b75392d\") " pod="openshift-dns/node-resolver-wfmns"
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034271    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-run-netns\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.035991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034281    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-run-ovn\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034297    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8cce14c3-bcec-404c-9873-9825555f2fc4-ovnkube-config\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034325    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lppgr\" (UniqueName: \"kubernetes.io/projected/8cce14c3-bcec-404c-9873-9825555f2fc4-kube-api-access-lppgr\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034352    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7kzw\" (UniqueName: \"kubernetes.io/projected/877fea60-1134-4e22-8db9-67000f4a4692-kube-api-access-t7kzw\") pod \"iptables-alerter-f4rtw\" (UID: \"877fea60-1134-4e22-8db9-67000f4a4692\") " pod="openshift-network-operator/iptables-alerter-f4rtw"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034388    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8cce14c3-bcec-404c-9873-9825555f2fc4-ovn-node-metrics-cert\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034413    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/877fea60-1134-4e22-8db9-67000f4a4692-host-slash\") pod \"iptables-alerter-f4rtw\" (UID: \"877fea60-1134-4e22-8db9-67000f4a4692\") " pod="openshift-network-operator/iptables-alerter-f4rtw"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034456    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx7hd\" (UniqueName: \"kubernetes.io/projected/fb05ec42-ba16-4d8b-82e1-1e689b75392d-kube-api-access-mx7hd\") pod \"node-resolver-wfmns\" (UID: \"fb05ec42-ba16-4d8b-82e1-1e689b75392d\") " pod="openshift-dns/node-resolver-wfmns"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034519    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/877fea60-1134-4e22-8db9-67000f4a4692-host-slash\") pod \"iptables-alerter-f4rtw\" (UID: \"877fea60-1134-4e22-8db9-67000f4a4692\") " pod="openshift-network-operator/iptables-alerter-f4rtw"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034548    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-kubelet\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034574    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-run-systemd\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034591    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a8a9f309-636c-41da-a9a7-6d33ebced403-kubelet-config\") pod \"global-pull-secret-syncer-bhrhl\" (UID: \"a8a9f309-636c-41da-a9a7-6d33ebced403\") " pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034600    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8cce14c3-bcec-404c-9873-9825555f2fc4-ovnkube-script-lib\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034636    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb05ec42-ba16-4d8b-82e1-1e689b75392d-tmp-dir\") pod \"node-resolver-wfmns\" (UID: \"fb05ec42-ba16-4d8b-82e1-1e689b75392d\") " pod="openshift-dns/node-resolver-wfmns"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034684    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cc015ede-26b1-49e2-9812-626d373e658f-serviceca\") pod \"node-ca-q9xvb\" (UID: \"cc015ede-26b1-49e2-9812-626d373e658f\") " pod="openshift-image-registry/node-ca-q9xvb"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034746    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-run-ovn-kubernetes\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034788    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q92jf\" (UniqueName: \"kubernetes.io/projected/cc015ede-26b1-49e2-9812-626d373e658f-kube-api-access-q92jf\") pod \"node-ca-q9xvb\" (UID: \"cc015ede-26b1-49e2-9812-626d373e658f\") " pod="openshift-image-registry/node-ca-q9xvb"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034819    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-node-log\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.036737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034858    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-cni-netd\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.037625 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034921    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/877fea60-1134-4e22-8db9-67000f4a4692-iptables-alerter-script\") pod \"iptables-alerter-f4rtw\" (UID: \"877fea60-1134-4e22-8db9-67000f4a4692\") " pod="openshift-network-operator/iptables-alerter-f4rtw"
Jul 02 13:14:11.037625 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034932    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-kubelet\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.037625 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034949    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8cce14c3-bcec-404c-9873-9825555f2fc4-env-overrides\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.037625 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.034984    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-run-netns\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.037625 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.035209    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8cce14c3-bcec-404c-9873-9825555f2fc4-ovnkube-script-lib\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.037625 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.035270    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-run-ovn-kubernetes\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.037625 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.035275    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-node-log\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.037625 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.035308    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-host-cni-netd\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.037625 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.035337    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8cce14c3-bcec-404c-9873-9825555f2fc4-run-systemd\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.037625 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.035646    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8cce14c3-bcec-404c-9873-9825555f2fc4-ovnkube-config\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.037625 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.035664    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/877fea60-1134-4e22-8db9-67000f4a4692-iptables-alerter-script\") pod \"iptables-alerter-f4rtw\" (UID: \"877fea60-1134-4e22-8db9-67000f4a4692\") " pod="openshift-network-operator/iptables-alerter-f4rtw"
Jul 02 13:14:11.037625 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.035650    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cc015ede-26b1-49e2-9812-626d373e658f-serviceca\") pod \"node-ca-q9xvb\" (UID: \"cc015ede-26b1-49e2-9812-626d373e658f\") " pod="openshift-image-registry/node-ca-q9xvb"
Jul 02 13:14:11.037625 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.036022    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8cce14c3-bcec-404c-9873-9825555f2fc4-env-overrides\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.038517 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.038493    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8cce14c3-bcec-404c-9873-9825555f2fc4-ovn-node-metrics-cert\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.042999 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.042952    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx7hd\" (UniqueName: \"kubernetes.io/projected/fb05ec42-ba16-4d8b-82e1-1e689b75392d-kube-api-access-mx7hd\") pod \"node-resolver-wfmns\" (UID: \"fb05ec42-ba16-4d8b-82e1-1e689b75392d\") " pod="openshift-dns/node-resolver-wfmns"
Jul 02 13:14:11.043114 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.043064    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7kzw\" (UniqueName: \"kubernetes.io/projected/877fea60-1134-4e22-8db9-67000f4a4692-kube-api-access-t7kzw\") pod \"iptables-alerter-f4rtw\" (UID: \"877fea60-1134-4e22-8db9-67000f4a4692\") " pod="openshift-network-operator/iptables-alerter-f4rtw"
Jul 02 13:14:11.044165 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.043837    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q92jf\" (UniqueName: \"kubernetes.io/projected/cc015ede-26b1-49e2-9812-626d373e658f-kube-api-access-q92jf\") pod \"node-ca-q9xvb\" (UID: \"cc015ede-26b1-49e2-9812-626d373e658f\") " pod="openshift-image-registry/node-ca-q9xvb"
Jul 02 13:14:11.044328 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.044280    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lppgr\" (UniqueName: \"kubernetes.io/projected/8cce14c3-bcec-404c-9873-9825555f2fc4-kube-api-access-lppgr\") pod \"ovnkube-node-djnhg\" (UID: \"8cce14c3-bcec-404c-9873-9825555f2fc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.108953 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.108635    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w"
Jul 02 13:14:11.117618 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.117588    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zwphv"
Jul 02 13:14:11.124593 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:11.124559    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2bd9d19_8c80_4487_b0c9_cf470b482741.slice/crio-635ce0fc099ea90fbab1c6f21d7e7430321cb0da5d161bd06e4cdc2ba3ff54bc WatchSource:0}: Error finding container 635ce0fc099ea90fbab1c6f21d7e7430321cb0da5d161bd06e4cdc2ba3ff54bc: Status 404 returned error can't find the container with id 635ce0fc099ea90fbab1c6f21d7e7430321cb0da5d161bd06e4cdc2ba3ff54bc
Jul 02 13:14:11.129745 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.129573    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tt9pg"
Jul 02 13:14:11.135549 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:11.135522    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11cd0142_78dd_4651_b2af_2c26f1ce70e4.slice/crio-003dde916f41db2339eab8b1ea73ea1dbcd68a0297203ae3a92def3625d015dd WatchSource:0}: Error finding container 003dde916f41db2339eab8b1ea73ea1dbcd68a0297203ae3a92def3625d015dd: Status 404 returned error can't find the container with id 003dde916f41db2339eab8b1ea73ea1dbcd68a0297203ae3a92def3625d015dd
Jul 02 13:14:11.135653 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.135600    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wfmns"
Jul 02 13:14:11.143277 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.143255    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9z5jl"
Jul 02 13:14:11.148228 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:11.148204    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9325a868_a924_4308_8533_25b57f13535a.slice/crio-c2ea7baaa78548e153c4d80f87387f0aa2d8b4cb4f59f60fc038dc97be06d0a5 WatchSource:0}: Error finding container c2ea7baaa78548e153c4d80f87387f0aa2d8b4cb4f59f60fc038dc97be06d0a5: Status 404 returned error can't find the container with id c2ea7baaa78548e153c4d80f87387f0aa2d8b4cb4f59f60fc038dc97be06d0a5
Jul 02 13:14:11.149783 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.149761    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g4qzs"
Jul 02 13:14:11.158267 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.158242    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f4rtw"
Jul 02 13:14:11.164614 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.164358    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:11.165274 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:11.165238    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9454c557_e08b_4fbd_92fe_a2cffddf034b.slice/crio-a410c5105a7d9f09875b307962632958f1e7da14df6f9a83dfba3b9f47f8a74a WatchSource:0}: Error finding container a410c5105a7d9f09875b307962632958f1e7da14df6f9a83dfba3b9f47f8a74a: Status 404 returned error can't find the container with id a410c5105a7d9f09875b307962632958f1e7da14df6f9a83dfba3b9f47f8a74a
Jul 02 13:14:11.168204 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.168180    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-q9xvb"
Jul 02 13:14:11.437662 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.437571    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs\") pod \"network-metrics-daemon-6nhcs\" (UID: \"aa2c8da0-b138-46ef-b9eb-10f6922697be\") " pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:11.437829 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:11.437700    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:14:11.437829 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:11.437755    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs podName:aa2c8da0-b138-46ef-b9eb-10f6922697be nodeName:}" failed. No retries permitted until 2026-07-02 13:14:12.437739196 +0000 UTC m=+4.060532098 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs") pod "network-metrics-daemon-6nhcs" (UID: "aa2c8da0-b138-46ef-b9eb-10f6922697be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:14:11.538690 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.538652    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret\") pod \"global-pull-secret-syncer-bhrhl\" (UID: \"a8a9f309-636c-41da-a9a7-6d33ebced403\") " pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:11.538884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.538711    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmxvp\" (UniqueName: \"kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp\") pod \"network-check-target-kv69m\" (UID: \"c41c0fa4-04c6-4e1b-9af2-0635451f3f26\") " pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:11.538884 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:11.538834    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:14:11.538884 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:11.538848    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:14:11.538884 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:11.538864    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:14:11.538884 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:11.538876    2570 projected.go:196] Error preparing data for projected volume kube-api-access-pmxvp for pod openshift-network-diagnostics/network-check-target-kv69m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:14:11.539115 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:11.538902    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret podName:a8a9f309-636c-41da-a9a7-6d33ebced403 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:12.538882476 +0000 UTC m=+4.161675386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret") pod "global-pull-secret-syncer-bhrhl" (UID: "a8a9f309-636c-41da-a9a7-6d33ebced403") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:14:11.539115 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:11.538921    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp podName:c41c0fa4-04c6-4e1b-9af2-0635451f3f26 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:12.538912288 +0000 UTC m=+4.161705176 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pmxvp" (UniqueName: "kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp") pod "network-check-target-kv69m" (UID: "c41c0fa4-04c6-4e1b-9af2-0635451f3f26") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:14:11.891361 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.891274    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 13:09:11 +0000 UTC" deadline="2027-12-22 12:52:52.246091082 +0000 UTC"
Jul 02 13:14:11.891361 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.891355    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12911h38m40.354740643s"
Jul 02 13:14:11.892197 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.891657    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:11.892197 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.891940    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:11.892197 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:11.892024    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:11.892534 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:11.892505    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:11.904768 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.904662    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-q9xvb" event={"ID":"cc015ede-26b1-49e2-9812-626d373e658f","Type":"ContainerStarted","Data":"46bce011a96bfc240c4e59e233d40cb7c69a3ace8a436f7bee6722e99d5931ce"}
Jul 02 13:14:11.908644 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.908503    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg" event={"ID":"8cce14c3-bcec-404c-9873-9825555f2fc4","Type":"ContainerStarted","Data":"c52e222e938d9387325c9d4f7d206d727362baa231ef0879c96c388a92d7d16b"}
Jul 02 13:14:11.911720 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.911630    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g4qzs" event={"ID":"9454c557-e08b-4fbd-92fe-a2cffddf034b","Type":"ContainerStarted","Data":"a410c5105a7d9f09875b307962632958f1e7da14df6f9a83dfba3b9f47f8a74a"}
Jul 02 13:14:11.913149 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.913120    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9z5jl" event={"ID":"ae596d11-edc3-4966-abdb-f76a7b0f9c94","Type":"ContainerStarted","Data":"01058cd65a474377e0da135e89c5f9d1df0875f25e69d0b2efd3bf5c366bd33a"}
Jul 02 13:14:11.921005 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.920972    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tt9pg" event={"ID":"9325a868-a924-4308-8533-25b57f13535a","Type":"ContainerStarted","Data":"c2ea7baaa78548e153c4d80f87387f0aa2d8b4cb4f59f60fc038dc97be06d0a5"}
Jul 02 13:14:11.924736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.924707    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwphv" event={"ID":"11cd0142-78dd-4651-b2af-2c26f1ce70e4","Type":"ContainerStarted","Data":"003dde916f41db2339eab8b1ea73ea1dbcd68a0297203ae3a92def3625d015dd"}
Jul 02 13:14:11.927410 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.927323    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f4rtw" event={"ID":"877fea60-1134-4e22-8db9-67000f4a4692","Type":"ContainerStarted","Data":"9db85d06240e554768b9e62838fff58ff1695f58008f12716594b6f98da0b062"}
Jul 02 13:14:11.931259 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.931192    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wfmns" event={"ID":"fb05ec42-ba16-4d8b-82e1-1e689b75392d","Type":"ContainerStarted","Data":"42cdc7c0febe7c18edb41b61489041ffe1faae7326984eefbb777bde1645e921"}
Jul 02 13:14:11.933070 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:11.933043    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w" event={"ID":"b2bd9d19-8c80-4487-b0c9-cf470b482741","Type":"ContainerStarted","Data":"635ce0fc099ea90fbab1c6f21d7e7430321cb0da5d161bd06e4cdc2ba3ff54bc"}
Jul 02 13:14:12.446547 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:12.446502    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs\") pod \"network-metrics-daemon-6nhcs\" (UID: \"aa2c8da0-b138-46ef-b9eb-10f6922697be\") " pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:12.446751 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:12.446674    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:14:12.446751 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:12.446742    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs podName:aa2c8da0-b138-46ef-b9eb-10f6922697be nodeName:}" failed. No retries permitted until 2026-07-02 13:14:14.446722021 +0000 UTC m=+6.069514924 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs") pod "network-metrics-daemon-6nhcs" (UID: "aa2c8da0-b138-46ef-b9eb-10f6922697be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:14:12.547480 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:12.547245    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmxvp\" (UniqueName: \"kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp\") pod \"network-check-target-kv69m\" (UID: \"c41c0fa4-04c6-4e1b-9af2-0635451f3f26\") " pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:12.547480 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:12.547343    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret\") pod \"global-pull-secret-syncer-bhrhl\" (UID: \"a8a9f309-636c-41da-a9a7-6d33ebced403\") " pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:12.547730 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:12.547494    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:14:12.547730 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:12.547562    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret podName:a8a9f309-636c-41da-a9a7-6d33ebced403 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:14.547543671 +0000 UTC m=+6.170336571 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret") pod "global-pull-secret-syncer-bhrhl" (UID: "a8a9f309-636c-41da-a9a7-6d33ebced403") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:14:12.548136 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:12.548008    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:14:12.548136 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:12.548034    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:14:12.548136 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:12.548048    2570 projected.go:196] Error preparing data for projected volume kube-api-access-pmxvp for pod openshift-network-diagnostics/network-check-target-kv69m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:14:12.548136 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:12.548096    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp podName:c41c0fa4-04c6-4e1b-9af2-0635451f3f26 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:14.54808125 +0000 UTC m=+6.170874155 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pmxvp" (UniqueName: "kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp") pod "network-check-target-kv69m" (UID: "c41c0fa4-04c6-4e1b-9af2-0635451f3f26") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:14:12.892549 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:12.891529    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 13:09:11 +0000 UTC" deadline="2028-01-02 16:14:16.743306905 +0000 UTC"
Jul 02 13:14:12.892549 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:12.891571    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13179h0m3.85174107s"
Jul 02 13:14:12.894516 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:12.893307    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:12.894516 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:12.893466    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:13.892206 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:13.892118    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:13.892206 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:13.892143    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:13.892428 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:13.892280    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:13.892428 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:13.892374    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:14.464005 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:14.463962    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs\") pod \"network-metrics-daemon-6nhcs\" (UID: \"aa2c8da0-b138-46ef-b9eb-10f6922697be\") " pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:14.464546 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:14.464137    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:14:14.464546 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:14.464211    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs podName:aa2c8da0-b138-46ef-b9eb-10f6922697be nodeName:}" failed. No retries permitted until 2026-07-02 13:14:18.464188373 +0000 UTC m=+10.086981281 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs") pod "network-metrics-daemon-6nhcs" (UID: "aa2c8da0-b138-46ef-b9eb-10f6922697be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:14:14.565203 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:14.565154    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret\") pod \"global-pull-secret-syncer-bhrhl\" (UID: \"a8a9f309-636c-41da-a9a7-6d33ebced403\") " pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:14.565382 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:14.565231    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmxvp\" (UniqueName: \"kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp\") pod \"network-check-target-kv69m\" (UID: \"c41c0fa4-04c6-4e1b-9af2-0635451f3f26\") " pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:14.565440 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:14.565411    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:14:14.565579 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:14.565452    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:14:14.565579 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:14.565468    2570 projected.go:196] Error preparing data for projected volume kube-api-access-pmxvp for pod openshift-network-diagnostics/network-check-target-kv69m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:14:14.565579 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:14.565534    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp podName:c41c0fa4-04c6-4e1b-9af2-0635451f3f26 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:18.565515615 +0000 UTC m=+10.188308526 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pmxvp" (UniqueName: "kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp") pod "network-check-target-kv69m" (UID: "c41c0fa4-04c6-4e1b-9af2-0635451f3f26") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:14:14.566003 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:14.565982    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:14:14.566103 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:14.566044    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret podName:a8a9f309-636c-41da-a9a7-6d33ebced403 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:18.566027658 +0000 UTC m=+10.188820564 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret") pod "global-pull-secret-syncer-bhrhl" (UID: "a8a9f309-636c-41da-a9a7-6d33ebced403") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:14:14.892884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:14.892796    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:14.893041 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:14.892954    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:15.893211 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:15.892790    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:15.893211 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:15.892933    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:15.893211 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:15.893032    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:15.893211 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:15.893117    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:16.893498 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:16.892311    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:16.893498 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:16.892469    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:17.892727 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:17.892676    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:17.892934 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:17.892862    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:17.892934 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:17.892875    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:17.893059 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:17.892992    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:18.498155 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:18.498059    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs\") pod \"network-metrics-daemon-6nhcs\" (UID: \"aa2c8da0-b138-46ef-b9eb-10f6922697be\") " pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:18.498640 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:18.498189    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:14:18.498640 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:18.498256    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs podName:aa2c8da0-b138-46ef-b9eb-10f6922697be nodeName:}" failed. No retries permitted until 2026-07-02 13:14:26.498239329 +0000 UTC m=+18.121032231 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs") pod "network-metrics-daemon-6nhcs" (UID: "aa2c8da0-b138-46ef-b9eb-10f6922697be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:14:18.598963 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:18.598924    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret\") pod \"global-pull-secret-syncer-bhrhl\" (UID: \"a8a9f309-636c-41da-a9a7-6d33ebced403\") " pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:18.599127 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:18.599005    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmxvp\" (UniqueName: \"kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp\") pod \"network-check-target-kv69m\" (UID: \"c41c0fa4-04c6-4e1b-9af2-0635451f3f26\") " pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:18.599127 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:18.599087    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:14:18.599229 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:18.599157    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret podName:a8a9f309-636c-41da-a9a7-6d33ebced403 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:26.599138279 +0000 UTC m=+18.221931176 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret") pod "global-pull-secret-syncer-bhrhl" (UID: "a8a9f309-636c-41da-a9a7-6d33ebced403") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:14:18.599229 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:18.599195    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:14:18.599229 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:18.599212    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:14:18.599229 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:18.599221    2570 projected.go:196] Error preparing data for projected volume kube-api-access-pmxvp for pod openshift-network-diagnostics/network-check-target-kv69m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:14:18.599475 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:18.599263    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp podName:c41c0fa4-04c6-4e1b-9af2-0635451f3f26 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:26.599249206 +0000 UTC m=+18.222042103 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pmxvp" (UniqueName: "kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp") pod "network-check-target-kv69m" (UID: "c41c0fa4-04c6-4e1b-9af2-0635451f3f26") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:14:18.893852 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:18.893354    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:18.893852 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:18.893504    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:19.892811 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:19.891909    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:19.892811 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:19.892298    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:19.892811 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:19.892098    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:19.892811 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:19.892769    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:20.892221 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:20.892190    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:20.892406 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:20.892341    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:21.891710 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:21.891677    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:21.891710 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:21.891695    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:21.892197 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:21.891803    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:21.892197 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:21.891864    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:22.892767 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:22.892733    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:22.893235 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:22.892878    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:23.892251 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:23.892210    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:23.892251 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:23.892251    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:23.892516 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:23.892346    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:23.892516 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:23.892502    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:24.892463 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:24.892427    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:24.892924 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:24.892556    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:25.892243 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:25.892210    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:25.892480 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:25.892246    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:25.892480 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:25.892342    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:25.892480 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:25.892464    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:26.556851 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:26.556811    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs\") pod \"network-metrics-daemon-6nhcs\" (UID: \"aa2c8da0-b138-46ef-b9eb-10f6922697be\") " pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:26.557035 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:26.556981    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:14:26.557098 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:26.557050    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs podName:aa2c8da0-b138-46ef-b9eb-10f6922697be nodeName:}" failed. No retries permitted until 2026-07-02 13:14:42.557032809 +0000 UTC m=+34.179825706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs") pod "network-metrics-daemon-6nhcs" (UID: "aa2c8da0-b138-46ef-b9eb-10f6922697be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:14:26.657539 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:26.657499    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret\") pod \"global-pull-secret-syncer-bhrhl\" (UID: \"a8a9f309-636c-41da-a9a7-6d33ebced403\") " pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:26.657726 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:26.657558    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmxvp\" (UniqueName: \"kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp\") pod \"network-check-target-kv69m\" (UID: \"c41c0fa4-04c6-4e1b-9af2-0635451f3f26\") " pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:26.657726 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:26.657675    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:14:26.657726 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:26.657704    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:14:26.657726 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:26.657716    2570 projected.go:196] Error preparing data for projected volume kube-api-access-pmxvp for pod openshift-network-diagnostics/network-check-target-kv69m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:14:26.657893 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:26.657763    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp podName:c41c0fa4-04c6-4e1b-9af2-0635451f3f26 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:42.657750345 +0000 UTC m=+34.280543242 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pmxvp" (UniqueName: "kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp") pod "network-check-target-kv69m" (UID: "c41c0fa4-04c6-4e1b-9af2-0635451f3f26") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:14:26.657893 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:26.657673    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:14:26.657893 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:26.657841    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret podName:a8a9f309-636c-41da-a9a7-6d33ebced403 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:42.657821878 +0000 UTC m=+34.280614777 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret") pod "global-pull-secret-syncer-bhrhl" (UID: "a8a9f309-636c-41da-a9a7-6d33ebced403") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:14:26.892840 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:26.892725    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:26.893268 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:26.892865    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:27.892049 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:27.892014    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:27.892244 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:27.892012    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:27.892244 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:27.892158    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:27.892244 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:27.892200    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:28.892939 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:28.892902    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:28.893409 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:28.893016    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:29.891704 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:29.891665    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:29.891879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:29.891665    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:29.891879 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:29.891822    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:29.891996 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:29.891884    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:30.892489 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:30.892465    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:30.892737 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:30.892615    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:30.972683 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:30.972660    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-238.ec2.internal" event={"ID":"10fd38b2c4f9fdad3bb40dd53988d42d","Type":"ContainerStarted","Data":"54279354c5ac6ab2bbdca7d9d48eb4582fcef1993bfd3fe73078913fe3c09043"}
Jul 02 13:14:31.003805 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.003756    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-238.ec2.internal" podStartSLOduration=21.003741023 podStartE2EDuration="21.003741023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:14:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:14:31.003422532 +0000 UTC m=+22.626215439" watchObservedRunningTime="2026-07-02 13:14:31.003741023 +0000 UTC m=+22.626533929"
Jul 02 13:14:31.891971 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.891630    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:31.892096 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.891664    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:31.892146 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:31.892093    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:31.892183 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:31.892157    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:31.975806 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.975776    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g4qzs" event={"ID":"9454c557-e08b-4fbd-92fe-a2cffddf034b","Type":"ContainerStarted","Data":"6508a08fbe883a7eb8c16f1e1e63a69602526939fafa5e17a4a5e6fbcb2134fe"}
Jul 02 13:14:31.977094 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.977071    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9z5jl" event={"ID":"ae596d11-edc3-4966-abdb-f76a7b0f9c94","Type":"ContainerStarted","Data":"1fc384872b28e8b83a61c083735cca1216bedff1000a4c45da2408440708791e"}
Jul 02 13:14:31.979709 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.979682    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tt9pg" event={"ID":"9325a868-a924-4308-8533-25b57f13535a","Type":"ContainerStarted","Data":"32670e8427192f7fd26d54d3c3039396f4ba67f337c0cc4029a79bf03acf9725"}
Jul 02 13:14:31.981815 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.981790    2570 generic.go:356] "Generic (PLEG): container finished" podID="11cd0142-78dd-4651-b2af-2c26f1ce70e4" containerID="1cd66df5998ee6060db98d33374ecc7e83ddb3f701b1968a0fa6f904b3522503" exitCode=0
Jul 02 13:14:31.981902 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.981845    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwphv" event={"ID":"11cd0142-78dd-4651-b2af-2c26f1ce70e4","Type":"ContainerDied","Data":"1cd66df5998ee6060db98d33374ecc7e83ddb3f701b1968a0fa6f904b3522503"}
Jul 02 13:14:31.983244 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.983226    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wfmns" event={"ID":"fb05ec42-ba16-4d8b-82e1-1e689b75392d","Type":"ContainerStarted","Data":"f5e87936d33d766b4085dd03f71eaed1d14351603fdba20f6cd97873c069f2dd"}
Jul 02 13:14:31.984733 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.984714    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w" event={"ID":"b2bd9d19-8c80-4487-b0c9-cf470b482741","Type":"ContainerStarted","Data":"c732a70341d0e2ea4501fd385c16ebea5274ca026ed8a36787d64c60744aafa7"}
Jul 02 13:14:31.986094 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.986072    2570 generic.go:356] "Generic (PLEG): container finished" podID="9f2dc82eac687745157e1911aad1aef8" containerID="8146106ffb559227bd0c8b685eec334372a84ae20a563dd67a193b9e6b51b09a" exitCode=0
Jul 02 13:14:31.986181 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.986139    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal" event={"ID":"9f2dc82eac687745157e1911aad1aef8","Type":"ContainerDied","Data":"8146106ffb559227bd0c8b685eec334372a84ae20a563dd67a193b9e6b51b09a"}
Jul 02 13:14:31.987549 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.987527    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-q9xvb" event={"ID":"cc015ede-26b1-49e2-9812-626d373e658f","Type":"ContainerStarted","Data":"953a1d17bfb275f4653e97038d66433a76fc2335491190a8b8597e43f4bfafea"}
Jul 02 13:14:31.989707 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.989689    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:14:31.990018 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.989999    2570 generic.go:356] "Generic (PLEG): container finished" podID="8cce14c3-bcec-404c-9873-9825555f2fc4" containerID="d45dd90e32a33024fa212ee74c6f138774deb6f092d8db3ae6b5eacdef5f5057" exitCode=1
Jul 02 13:14:31.990084 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.990064    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg" event={"ID":"8cce14c3-bcec-404c-9873-9825555f2fc4","Type":"ContainerStarted","Data":"15d671e22a973c1c26d607062c0c75a9dae82471865187995a251269980ca429"}
Jul 02 13:14:31.990130 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.990119    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg" event={"ID":"8cce14c3-bcec-404c-9873-9825555f2fc4","Type":"ContainerStarted","Data":"850bbb6c261280fe295698cf7326f8c29c938b6f61af5d9d239fc438df7761ea"}
Jul 02 13:14:31.990183 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.990172    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg" event={"ID":"8cce14c3-bcec-404c-9873-9825555f2fc4","Type":"ContainerDied","Data":"d45dd90e32a33024fa212ee74c6f138774deb6f092d8db3ae6b5eacdef5f5057"}
Jul 02 13:14:31.990219 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.990210    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg" event={"ID":"8cce14c3-bcec-404c-9873-9825555f2fc4","Type":"ContainerStarted","Data":"31cddd3b8a7c5f47c009456f7ee4ee4a093d2d3c457b406a9dbac1549af38829"}
Jul 02 13:14:31.992138 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:31.992104    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-g4qzs" podStartSLOduration=3.326356154 podStartE2EDuration="22.99209344s" totalImagesPullingTime="19.665737286s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:14:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:14:31.991925773 +0000 UTC m=+23.614718680" watchObservedRunningTime="2026-07-02 13:14:31.99209344 +0000 UTC m=+23.614886345"
Jul 02 13:14:32.021724 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:32.021671    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9z5jl" podStartSLOduration=8.154977883 podStartE2EDuration="23.021654138s" totalImagesPullingTime="14.866676255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:14:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:14:32.021015889 +0000 UTC m=+23.643808795" watchObservedRunningTime="2026-07-02 13:14:32.021654138 +0000 UTC m=+23.644447043"
Jul 02 13:14:32.034064 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:32.034015    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-q9xvb" podStartSLOduration=2.430623643 podStartE2EDuration="22.033995552s" totalImagesPullingTime="19.603371909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:14:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:14:32.033450152 +0000 UTC m=+23.656243060" watchObservedRunningTime="2026-07-02 13:14:32.033995552 +0000 UTC m=+23.656788459"
Jul 02 13:14:32.071923 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:32.071883    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tt9pg" podStartSLOduration=3.401437612 podStartE2EDuration="23.071866762s" totalImagesPullingTime="19.67042915s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:14:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:14:32.07150729 +0000 UTC m=+23.694300197" watchObservedRunningTime="2026-07-02 13:14:32.071866762 +0000 UTC m=+23.694659667"
Jul 02 13:14:32.091841 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:32.091794    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wfmns" podStartSLOduration=2.43225949 podStartE2EDuration="22.091779254s" totalImagesPullingTime="19.659519764s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:14:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:14:32.0914808 +0000 UTC m=+23.714273701" watchObservedRunningTime="2026-07-02 13:14:32.091779254 +0000 UTC m=+23.714572160"
Jul 02 13:14:32.891793 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:32.891761    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:32.891941 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:32.891917    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:32.972166 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:32.972139    2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 02 13:14:32.995097 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:32.995065    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f4rtw" event={"ID":"877fea60-1134-4e22-8db9-67000f4a4692","Type":"ContainerStarted","Data":"826cc9ed454ec2e6e78c364c1ce359646b1945215106c3a263de3d620157e042"}
Jul 02 13:14:32.997132 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:32.997097    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w" event={"ID":"b2bd9d19-8c80-4487-b0c9-cf470b482741","Type":"ContainerStarted","Data":"77d366736dfff33d722b52180e4a3029c3642e804af1a88738cf1dabfff9a098"}
Jul 02 13:14:32.999168 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:32.999141    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal" event={"ID":"9f2dc82eac687745157e1911aad1aef8","Type":"ContainerStarted","Data":"84ed24a9477760f342aacb69589e2e544b18ace45170ad689011843792edad17"}
Jul 02 13:14:33.003678 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:33.003656    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:14:33.004053 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:33.004031    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg" event={"ID":"8cce14c3-bcec-404c-9873-9825555f2fc4","Type":"ContainerStarted","Data":"d65986a9f739a4c0c7eefb2746790e70e13ede00df2dc19cd9728191f4d2d06c"}
Jul 02 13:14:33.004140 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:33.004109    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg" event={"ID":"8cce14c3-bcec-404c-9873-9825555f2fc4","Type":"ContainerStarted","Data":"d6d4772ca969e63b5776e9f079e8c0dbffba34c3587709a49af69a7efc6565dc"}
Jul 02 13:14:33.007186 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:33.007140    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f4rtw" podStartSLOduration=3.393628666 podStartE2EDuration="23.007124118s" totalImagesPullingTime="19.613495452s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:14:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:14:33.006683817 +0000 UTC m=+24.629476724" watchObservedRunningTime="2026-07-02 13:14:33.007124118 +0000 UTC m=+24.629917025"
Jul 02 13:14:33.022086 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:33.022041    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-238.ec2.internal" podStartSLOduration=23.022031014 podStartE2EDuration="23.022031014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:14:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:14:33.021701901 +0000 UTC m=+24.644494807" watchObservedRunningTime="2026-07-02 13:14:33.022031014 +0000 UTC m=+24.644823921"
Jul 02 13:14:33.039922 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:33.039795    2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T13:14:32.972155977Z","UUID":"1bf76295-414a-4357-b082-72188862c071","Handler":null,"Name":"","Endpoint":""}
Jul 02 13:14:33.041419 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:33.041396    2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 02 13:14:33.041581 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:33.041445    2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 02 13:14:33.891902 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:33.891833    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:33.892164 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:33.891960    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:33.892164 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:33.892018    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:33.892164 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:33.892138    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:34.892894 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:34.892688    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:34.893332 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:34.892960    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:35.012746 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:35.012715    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:14:35.013188 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:35.013155    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg" event={"ID":"8cce14c3-bcec-404c-9873-9825555f2fc4","Type":"ContainerStarted","Data":"2f85fed4f629182c4829dacf55ee8d778a1b20c354ec3dcffd30a3c0fdb2062c"}
Jul 02 13:14:35.019004 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:35.018970    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w" event={"ID":"b2bd9d19-8c80-4487-b0c9-cf470b482741","Type":"ContainerStarted","Data":"dcd8636ff5ffb7fdb3ff7e38b39200855f74670bcb0d6233283ccfd7624759d0"}
Jul 02 13:14:35.033486 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:35.033432    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pdq9w" podStartSLOduration=3.761605152 podStartE2EDuration="26.033411635s" totalImagesPullingTime="22.271806483s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:14:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:14:35.033302543 +0000 UTC m=+26.656095450" watchObservedRunningTime="2026-07-02 13:14:35.033411635 +0000 UTC m=+26.656204539"
Jul 02 13:14:35.891709 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:35.891676    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:35.891709 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:35.891702    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:35.891962 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:35.891803    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:35.891962 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:35.891918    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:35.968314 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:35.968275    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9z5jl"
Jul 02 13:14:35.969018 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:35.968880    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9z5jl"
Jul 02 13:14:36.021154 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:36.021107    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9z5jl"
Jul 02 13:14:36.021467 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:36.021447    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9z5jl"
Jul 02 13:14:36.892573 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:36.892400    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:36.892700 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:36.892673    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:37.026619 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:37.026563    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:14:37.027184 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:37.026943    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg" event={"ID":"8cce14c3-bcec-404c-9873-9825555f2fc4","Type":"ContainerStarted","Data":"d235638b5824b47f47b75076c4bae350e6a949e1e4f36c674f91a38bfcd73978"}
Jul 02 13:14:37.027243 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:37.027230    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:37.027298 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:37.027290    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:37.027478 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:37.027460    2570 scope.go:117] "RemoveContainer" containerID="d45dd90e32a33024fa212ee74c6f138774deb6f092d8db3ae6b5eacdef5f5057"
Jul 02 13:14:37.028849 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:37.028828    2570 generic.go:356] "Generic (PLEG): container finished" podID="11cd0142-78dd-4651-b2af-2c26f1ce70e4" containerID="b82ad7e04567f3e8c7cd537e072849a4f24a90b3bfd94ecf4a90f0c5e5151641" exitCode=0
Jul 02 13:14:37.028945 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:37.028920    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwphv" event={"ID":"11cd0142-78dd-4651-b2af-2c26f1ce70e4","Type":"ContainerDied","Data":"b82ad7e04567f3e8c7cd537e072849a4f24a90b3bfd94ecf4a90f0c5e5151641"}
Jul 02 13:14:37.043717 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:37.043624    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:37.892098 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:37.892076    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:37.892231 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:37.892076    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:37.892231 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:37.892174    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:37.892310 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:37.892257    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:38.033401 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:38.033358    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwphv" event={"ID":"11cd0142-78dd-4651-b2af-2c26f1ce70e4","Type":"ContainerStarted","Data":"433df3dd0ecd867ee7348722b07885df78044368b68748702535775231eb2141"}
Jul 02 13:14:38.037000 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:38.036979    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:14:38.037362 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:38.037336    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg" event={"ID":"8cce14c3-bcec-404c-9873-9825555f2fc4","Type":"ContainerStarted","Data":"17e11e1339556033a7e931229caec758a4103e952cc33a53fdb64ac19428610d"}
Jul 02 13:14:38.037573 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:38.037555    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:38.055108 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:38.055083    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:14:38.074118 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:38.074071    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg" podStartSLOduration=7.837516578 podStartE2EDuration="28.074053555s" totalImagesPullingTime="20.236536977s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:14:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:14:38.073487414 +0000 UTC m=+29.696280321" watchObservedRunningTime="2026-07-02 13:14:38.074053555 +0000 UTC m=+29.696846460"
Jul 02 13:14:38.532562 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:38.532376    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bhrhl"]
Jul 02 13:14:38.532741 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:38.532692    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:38.532832 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:38.532808    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:38.535972 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:38.535934    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6nhcs"]
Jul 02 13:14:38.536107 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:38.536064    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:38.536167 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:38.536148    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:38.536699 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:38.536676    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kv69m"]
Jul 02 13:14:38.536805 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:38.536775    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:38.536878 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:38.536859    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:39.041271 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:39.041242    2570 generic.go:356] "Generic (PLEG): container finished" podID="11cd0142-78dd-4651-b2af-2c26f1ce70e4" containerID="433df3dd0ecd867ee7348722b07885df78044368b68748702535775231eb2141" exitCode=0
Jul 02 13:14:39.041670 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:39.041331    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwphv" event={"ID":"11cd0142-78dd-4651-b2af-2c26f1ce70e4","Type":"ContainerDied","Data":"433df3dd0ecd867ee7348722b07885df78044368b68748702535775231eb2141"}
Jul 02 13:14:39.891909 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:39.891871    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:39.892128 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:39.891975    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:39.892128 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:39.891986    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:39.892128 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:39.892007    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:39.892128 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:39.892103    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:39.892311 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:39.892200    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:41.048892 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:41.048851    2570 generic.go:356] "Generic (PLEG): container finished" podID="11cd0142-78dd-4651-b2af-2c26f1ce70e4" containerID="b386d460eba4ae938d56a6ddb4ce6d6b0e93ed2425a67ea186b06ba5940520cf" exitCode=0
Jul 02 13:14:41.049539 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:41.048907    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwphv" event={"ID":"11cd0142-78dd-4651-b2af-2c26f1ce70e4","Type":"ContainerDied","Data":"b386d460eba4ae938d56a6ddb4ce6d6b0e93ed2425a67ea186b06ba5940520cf"}
Jul 02 13:14:41.891939 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:41.891902    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:41.891939 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:41.891925    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:41.891939 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:41.891925    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:41.892233 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:41.892032    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:41.892233 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:41.892163    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:41.892314 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:41.892244    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:42.580436 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:42.580397    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs\") pod \"network-metrics-daemon-6nhcs\" (UID: \"aa2c8da0-b138-46ef-b9eb-10f6922697be\") " pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:42.580989 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:42.580517    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:14:42.580989 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:42.580590    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs podName:aa2c8da0-b138-46ef-b9eb-10f6922697be nodeName:}" failed. No retries permitted until 2026-07-02 13:15:14.580571624 +0000 UTC m=+66.203364528 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs") pod "network-metrics-daemon-6nhcs" (UID: "aa2c8da0-b138-46ef-b9eb-10f6922697be") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:14:42.681530 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:42.681498    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmxvp\" (UniqueName: \"kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp\") pod \"network-check-target-kv69m\" (UID: \"c41c0fa4-04c6-4e1b-9af2-0635451f3f26\") " pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:42.681700 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:42.681567    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret\") pod \"global-pull-secret-syncer-bhrhl\" (UID: \"a8a9f309-636c-41da-a9a7-6d33ebced403\") " pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:42.681700 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:42.681684    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:14:42.681803 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:42.681711    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:14:42.681803 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:42.681724    2570 projected.go:196] Error preparing data for projected volume kube-api-access-pmxvp for pod openshift-network-diagnostics/network-check-target-kv69m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:14:42.681803 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:42.681686    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:14:42.681894 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:42.681787    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp podName:c41c0fa4-04c6-4e1b-9af2-0635451f3f26 nodeName:}" failed. No retries permitted until 2026-07-02 13:15:14.681768703 +0000 UTC m=+66.304561610 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pmxvp" (UniqueName: "kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp") pod "network-check-target-kv69m" (UID: "c41c0fa4-04c6-4e1b-9af2-0635451f3f26") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:14:42.681894 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:42.681849    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret podName:a8a9f309-636c-41da-a9a7-6d33ebced403 nodeName:}" failed. No retries permitted until 2026-07-02 13:15:14.681832276 +0000 UTC m=+66.304625174 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret") pod "global-pull-secret-syncer-bhrhl" (UID: "a8a9f309-636c-41da-a9a7-6d33ebced403") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:14:43.892560 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:43.892357    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:43.892979 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:43.892364    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:43.892979 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:43.892651    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bhrhl" podUID="a8a9f309-636c-41da-a9a7-6d33ebced403"
Jul 02 13:14:43.892979 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:43.892364    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:43.892979 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:43.892732    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kv69m" podUID="c41c0fa4-04c6-4e1b-9af2-0635451f3f26"
Jul 02 13:14:43.892979 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:43.892808    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:14:45.189685 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.189569    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-238.ec2.internal" event="NodeReady"
Jul 02 13:14:45.190124 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.189749    2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 02 13:14:45.221468 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.221431    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-f7c8df46f-cdx4k"]
Jul 02 13:14:45.227097 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.227067    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"]
Jul 02 13:14:45.227258 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.227207    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.229147 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.229122    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 02 13:14:45.229306 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.229191    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 02 13:14:45.229306 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.229125    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 02 13:14:45.229306 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.229194    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wn2v6\""
Jul 02 13:14:45.230486 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.230467    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:14:45.232080 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.232060    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 02 13:14:45.232244 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.232200    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-vs9mx\""
Jul 02 13:14:45.232325 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.232241    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 02 13:14:45.234998 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.234962    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 02 13:14:45.240077 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.240057    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"]
Jul 02 13:14:45.241497 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.241477    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f7c8df46f-cdx4k"]
Jul 02 13:14:45.242729 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.242706    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-h66tv"]
Jul 02 13:14:45.246480 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.246459    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dfhv6"]
Jul 02 13:14:45.246604 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.246587    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h66tv"
Jul 02 13:14:45.248487 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.248467    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 02 13:14:45.248591 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.248492    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 02 13:14:45.248591 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.248533    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d2c87\""
Jul 02 13:14:45.250906 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.250888    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dfhv6"
Jul 02 13:14:45.252468 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.252316    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h66tv"]
Jul 02 13:14:45.252831 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.252810    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nvh8q\""
Jul 02 13:14:45.252979 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.252958    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dfhv6"]
Jul 02 13:14:45.253143 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.253126    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 02 13:14:45.253284 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.253260    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 02 13:14:45.253346 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.253289    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 02 13:14:45.366852 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.366787    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:45.401616 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.401584    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvk4l\" (UniqueName: \"kubernetes.io/projected/6b13232c-9bcc-423b-a85f-1ef580a1d477-kube-api-access-vvk4l\") pod \"ingress-canary-dfhv6\" (UID: \"6b13232c-9bcc-423b-a85f-1ef580a1d477\") " pod="openshift-ingress-canary/ingress-canary-dfhv6"
Jul 02 13:14:45.401795 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.401646    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-installation-pull-secrets\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.401795 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.401755    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-image-registry-private-configuration\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.401795 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.401782    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-ca-trust-extracted\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.401968 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.401812    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-trusted-ca\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.401968 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.401838    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:14:45.401968 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.401859    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0971a04b-36bc-46ca-9bce-762b11f50d31-tmp-dir\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:14:45.401968 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.401898    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kdlkr\" (UID: \"b61b8094-e8b8-406b-89db-3c580535536d\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:14:45.401968 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.401923    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-bound-sa-token\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.402213 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.401972    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0971a04b-36bc-46ca-9bce-762b11f50d31-config-volume\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:14:45.402213 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.401990    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.402213 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.402011    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prztc\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-kube-api-access-prztc\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.402213 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.402037    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dk8h\" (UniqueName: \"kubernetes.io/projected/0971a04b-36bc-46ca-9bce-762b11f50d31-kube-api-access-7dk8h\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:14:45.402213 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.402060    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b61b8094-e8b8-406b-89db-3c580535536d-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-kdlkr\" (UID: \"b61b8094-e8b8-406b-89db-3c580535536d\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:14:45.402213 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.402082    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-certificates\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.402213 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.402100    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b13232c-9bcc-423b-a85f-1ef580a1d477-cert\") pod \"ingress-canary-dfhv6\" (UID: \"6b13232c-9bcc-423b-a85f-1ef580a1d477\") " pod="openshift-ingress-canary/ingress-canary-dfhv6"
Jul 02 13:14:45.457093 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.457037    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:45.502852 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.502819    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0971a04b-36bc-46ca-9bce-762b11f50d31-config-volume\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:14:45.503011 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.502864    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.503011 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.502896    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prztc\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-kube-api-access-prztc\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.503011 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.502923    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dk8h\" (UniqueName: \"kubernetes.io/projected/0971a04b-36bc-46ca-9bce-762b11f50d31-kube-api-access-7dk8h\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:14:45.503011 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.502948    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b61b8094-e8b8-406b-89db-3c580535536d-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-kdlkr\" (UID: \"b61b8094-e8b8-406b-89db-3c580535536d\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:14:45.503011 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.502969    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-certificates\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.503011 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.502993    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b13232c-9bcc-423b-a85f-1ef580a1d477-cert\") pod \"ingress-canary-dfhv6\" (UID: \"6b13232c-9bcc-423b-a85f-1ef580a1d477\") " pod="openshift-ingress-canary/ingress-canary-dfhv6"
Jul 02 13:14:45.503284 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.503019    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvk4l\" (UniqueName: \"kubernetes.io/projected/6b13232c-9bcc-423b-a85f-1ef580a1d477-kube-api-access-vvk4l\") pod \"ingress-canary-dfhv6\" (UID: \"6b13232c-9bcc-423b-a85f-1ef580a1d477\") " pod="openshift-ingress-canary/ingress-canary-dfhv6"
Jul 02 13:14:45.503284 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:45.503032    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 13:14:45.503284 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:45.503054    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f7c8df46f-cdx4k: secret "image-registry-tls" not found
Jul 02 13:14:45.503284 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.503085    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-installation-pull-secrets\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.503284 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:45.503116    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls podName:bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:46.003095442 +0000 UTC m=+37.625888344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls") pod "image-registry-f7c8df46f-cdx4k" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7") : secret "image-registry-tls" not found
Jul 02 13:14:45.503284 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.503184    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-image-registry-private-configuration\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.503284 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.503216    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-ca-trust-extracted\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.503627 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.503294    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-trusted-ca\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.503627 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.503335    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:14:45.503627 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.503361    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0971a04b-36bc-46ca-9bce-762b11f50d31-tmp-dir\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:14:45.503627 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.503433    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kdlkr\" (UID: \"b61b8094-e8b8-406b-89db-3c580535536d\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:14:45.503627 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.503461    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-bound-sa-token\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.503627 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.503493    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0971a04b-36bc-46ca-9bce-762b11f50d31-config-volume\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:14:45.503893 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:45.503850    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 02 13:14:45.503938 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:45.503904    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert podName:b61b8094-e8b8-406b-89db-3c580535536d nodeName:}" failed. No retries permitted until 2026-07-02 13:14:46.003888541 +0000 UTC m=+37.626681439 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-kdlkr" (UID: "b61b8094-e8b8-406b-89db-3c580535536d") : secret "networking-console-plugin-cert" not found
Jul 02 13:14:45.503985 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.503958    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0971a04b-36bc-46ca-9bce-762b11f50d31-tmp-dir\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:14:45.504570 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.504523    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-ca-trust-extracted\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.504570 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.504542    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-certificates\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.504745 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:45.504617    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:14:45.504745 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:45.504671    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls podName:0971a04b-36bc-46ca-9bce-762b11f50d31 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:46.004655638 +0000 UTC m=+37.627448545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls") pod "dns-default-h66tv" (UID: "0971a04b-36bc-46ca-9bce-762b11f50d31") : secret "dns-default-metrics-tls" not found
Jul 02 13:14:45.504889 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.504861    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b61b8094-e8b8-406b-89db-3c580535536d-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-kdlkr\" (UID: \"b61b8094-e8b8-406b-89db-3c580535536d\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:14:45.505018 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.504963    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-trusted-ca\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.507973 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.507954    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-installation-pull-secrets\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.508046 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.507973    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-image-registry-private-configuration\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.508046 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.508014    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b13232c-9bcc-423b-a85f-1ef580a1d477-cert\") pod \"ingress-canary-dfhv6\" (UID: \"6b13232c-9bcc-423b-a85f-1ef580a1d477\") " pod="openshift-ingress-canary/ingress-canary-dfhv6"
Jul 02 13:14:45.512026 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.511981    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvk4l\" (UniqueName: \"kubernetes.io/projected/6b13232c-9bcc-423b-a85f-1ef580a1d477-kube-api-access-vvk4l\") pod \"ingress-canary-dfhv6\" (UID: \"6b13232c-9bcc-423b-a85f-1ef580a1d477\") " pod="openshift-ingress-canary/ingress-canary-dfhv6"
Jul 02 13:14:45.512026 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.512006    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-bound-sa-token\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.512177 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.512078    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dk8h\" (UniqueName: \"kubernetes.io/projected/0971a04b-36bc-46ca-9bce-762b11f50d31-kube-api-access-7dk8h\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:14:45.513415 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.513378    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prztc\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-kube-api-access-prztc\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:45.566655 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.566616    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dfhv6"
Jul 02 13:14:45.847255 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.847228    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dfhv6"]
Jul 02 13:14:45.853894 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:14:45.853866    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b13232c_9bcc_423b_a85f_1ef580a1d477.slice/crio-c28ff6004e92361d520524b9bbd1637e994c1cd97e6919820d222e4db5848566 WatchSource:0}: Error finding container c28ff6004e92361d520524b9bbd1637e994c1cd97e6919820d222e4db5848566: Status 404 returned error can't find the container with id c28ff6004e92361d520524b9bbd1637e994c1cd97e6919820d222e4db5848566
Jul 02 13:14:45.892030 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.891833    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:14:45.892080 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.891842    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:14:45.892159 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.891867    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:14:45.893965 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.893937    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 02 13:14:45.894081 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.893950    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 02 13:14:45.894242 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.894226    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 02 13:14:45.894313 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.894272    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t5cpn\""
Jul 02 13:14:45.894313 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.894305    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k9zxt\""
Jul 02 13:14:45.894431 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:45.894317    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 02 13:14:46.007721 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:46.007693    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:46.007854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:46.007772    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:14:46.007854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:46.007793    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kdlkr\" (UID: \"b61b8094-e8b8-406b-89db-3c580535536d\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:14:46.007854 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:46.007837    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 13:14:46.007854 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:46.007856    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f7c8df46f-cdx4k: secret "image-registry-tls" not found
Jul 02 13:14:46.008029 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:46.007876    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 02 13:14:46.008029 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:46.007913    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:14:46.008029 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:46.007917    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls podName:bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:47.007903018 +0000 UTC m=+38.630695918 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls") pod "image-registry-f7c8df46f-cdx4k" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7") : secret "image-registry-tls" not found
Jul 02 13:14:46.008029 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:46.007964    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert podName:b61b8094-e8b8-406b-89db-3c580535536d nodeName:}" failed. No retries permitted until 2026-07-02 13:14:47.007950854 +0000 UTC m=+38.630743738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-kdlkr" (UID: "b61b8094-e8b8-406b-89db-3c580535536d") : secret "networking-console-plugin-cert" not found
Jul 02 13:14:46.008029 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:46.007974    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls podName:0971a04b-36bc-46ca-9bce-762b11f50d31 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:47.007967872 +0000 UTC m=+38.630760757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls") pod "dns-default-h66tv" (UID: "0971a04b-36bc-46ca-9bce-762b11f50d31") : secret "dns-default-metrics-tls" not found
Jul 02 13:14:46.063563 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:46.063532    2570 generic.go:356] "Generic (PLEG): container finished" podID="11cd0142-78dd-4651-b2af-2c26f1ce70e4" containerID="8e3b4d0dcbadc1bcec2e63eb8140816d382137880f123f53785ed499645fa644" exitCode=0
Jul 02 13:14:46.063749 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:46.063589    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwphv" event={"ID":"11cd0142-78dd-4651-b2af-2c26f1ce70e4","Type":"ContainerDied","Data":"8e3b4d0dcbadc1bcec2e63eb8140816d382137880f123f53785ed499645fa644"}
Jul 02 13:14:46.064664 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:46.064642    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dfhv6" event={"ID":"6b13232c-9bcc-423b-a85f-1ef580a1d477","Type":"ContainerStarted","Data":"c28ff6004e92361d520524b9bbd1637e994c1cd97e6919820d222e4db5848566"}
Jul 02 13:14:46.147055 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:46.147007    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:46.233227 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:46.233148    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:46.336175 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:46.336138    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:47.016047 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:47.016007    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:14:47.016255 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:47.016060    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kdlkr\" (UID: \"b61b8094-e8b8-406b-89db-3c580535536d\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:14:47.016255 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:47.016124    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:47.016255 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:47.016185    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:14:47.016255 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:47.016241    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 02 13:14:47.016469 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:47.016270    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 13:14:47.016469 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:47.016284    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f7c8df46f-cdx4k: secret "image-registry-tls" not found
Jul 02 13:14:47.016469 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:47.016248    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls podName:0971a04b-36bc-46ca-9bce-762b11f50d31 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:49.016232084 +0000 UTC m=+40.639024991 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls") pod "dns-default-h66tv" (UID: "0971a04b-36bc-46ca-9bce-762b11f50d31") : secret "dns-default-metrics-tls" not found
Jul 02 13:14:47.016469 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:47.016316    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert podName:b61b8094-e8b8-406b-89db-3c580535536d nodeName:}" failed. No retries permitted until 2026-07-02 13:14:49.016298965 +0000 UTC m=+40.639091856 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-kdlkr" (UID: "b61b8094-e8b8-406b-89db-3c580535536d") : secret "networking-console-plugin-cert" not found
Jul 02 13:14:47.016469 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:47.016334    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls podName:bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:49.016322982 +0000 UTC m=+40.639115870 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls") pod "image-registry-f7c8df46f-cdx4k" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7") : secret "image-registry-tls" not found
Jul 02 13:14:47.070687 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:47.070650    2570 generic.go:356] "Generic (PLEG): container finished" podID="11cd0142-78dd-4651-b2af-2c26f1ce70e4" containerID="92be54bcdaadd86e1962d49d812cdc69100fb28fae14de304aab3f1da9649b34" exitCode=0
Jul 02 13:14:47.070831 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:47.070712    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwphv" event={"ID":"11cd0142-78dd-4651-b2af-2c26f1ce70e4","Type":"ContainerDied","Data":"92be54bcdaadd86e1962d49d812cdc69100fb28fae14de304aab3f1da9649b34"}
Jul 02 13:14:47.216845 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:47.216777    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:47.272699 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:47.272618    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:47.326051 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:47.326010    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:48.077358 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:48.077320    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwphv" event={"ID":"11cd0142-78dd-4651-b2af-2c26f1ce70e4","Type":"ContainerStarted","Data":"92fc2e6bc935b03299cbb12e1513bbf6511496dd31d2064e49a913c8995831df"}
Jul 02 13:14:48.096818 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:48.096765    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zwphv" podStartSLOduration=7.816468654 podStartE2EDuration="39.096748101s" totalImagesPullingTime="31.280279447s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:14:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:14:48.096020558 +0000 UTC m=+39.718813468" watchObservedRunningTime="2026-07-02 13:14:48.096748101 +0000 UTC m=+39.719541006"
Jul 02 13:14:48.219274 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:48.219234    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:48.269525 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:48.269484    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:48.537460 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:48.537428    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:49.036245 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:49.036210    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:49.036435 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:49.036312    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:14:49.036435 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:49.036344    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kdlkr\" (UID: \"b61b8094-e8b8-406b-89db-3c580535536d\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:14:49.036435 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:49.036372    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 13:14:49.036435 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:49.036411    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f7c8df46f-cdx4k: secret "image-registry-tls" not found
Jul 02 13:14:49.036608 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:49.036472    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls podName:bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:53.036452419 +0000 UTC m=+44.659245323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls") pod "image-registry-f7c8df46f-cdx4k" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7") : secret "image-registry-tls" not found
Jul 02 13:14:49.036608 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:49.036484    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 02 13:14:49.036608 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:49.036536    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert podName:b61b8094-e8b8-406b-89db-3c580535536d nodeName:}" failed. No retries permitted until 2026-07-02 13:14:53.036520044 +0000 UTC m=+44.659312948 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-kdlkr" (UID: "b61b8094-e8b8-406b-89db-3c580535536d") : secret "networking-console-plugin-cert" not found
Jul 02 13:14:49.036608 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:49.036536    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:14:49.036608 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:49.036573    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls podName:0971a04b-36bc-46ca-9bce-762b11f50d31 nodeName:}" failed. No retries permitted until 2026-07-02 13:14:53.036564585 +0000 UTC m=+44.659357485 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls") pod "dns-default-h66tv" (UID: "0971a04b-36bc-46ca-9bce-762b11f50d31") : secret "dns-default-metrics-tls" not found
Jul 02 13:14:49.080405 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:49.080315    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dfhv6" event={"ID":"6b13232c-9bcc-423b-a85f-1ef580a1d477","Type":"ContainerStarted","Data":"a584b58de4ae85d1b994eeb99b6280401f2794c0658f4865f0e384347d03d47c"}
Jul 02 13:14:49.095031 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:49.094984    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dfhv6" podStartSLOduration=1.252011983 podStartE2EDuration="4.094970787s" totalImagesPullingTime="2.842958804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:14:49.094862878 +0000 UTC m=+40.717655784" watchObservedRunningTime="2026-07-02 13:14:49.094970787 +0000 UTC m=+40.717763693"
Jul 02 13:14:52.748265 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:52.748170    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:52.800675 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:52.800617    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:53.065693 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:53.065607    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:14:53.065693 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:53.065646    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kdlkr\" (UID: \"b61b8094-e8b8-406b-89db-3c580535536d\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:14:53.065851 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:53.065744    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 02 13:14:53.065851 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:53.065749    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:14:53.065851 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:53.065776    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:14:53.065851 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:53.065798    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert podName:b61b8094-e8b8-406b-89db-3c580535536d nodeName:}" failed. No retries permitted until 2026-07-02 13:15:01.065784595 +0000 UTC m=+52.688577496 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-kdlkr" (UID: "b61b8094-e8b8-406b-89db-3c580535536d") : secret "networking-console-plugin-cert" not found
Jul 02 13:14:53.065851 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:53.065823    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls podName:0971a04b-36bc-46ca-9bce-762b11f50d31 nodeName:}" failed. No retries permitted until 2026-07-02 13:15:01.06581531 +0000 UTC m=+52.688608193 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls") pod "dns-default-h66tv" (UID: "0971a04b-36bc-46ca-9bce-762b11f50d31") : secret "dns-default-metrics-tls" not found
Jul 02 13:14:53.065851 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:53.065839    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 13:14:53.065851 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:53.065850    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f7c8df46f-cdx4k: secret "image-registry-tls" not found
Jul 02 13:14:53.066071 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:14:53.065892    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls podName:bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7 nodeName:}" failed. No retries permitted until 2026-07-02 13:15:01.065878214 +0000 UTC m=+52.688671114 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls") pod "image-registry-f7c8df46f-cdx4k" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7") : secret "image-registry-tls" not found
Jul 02 13:14:53.940010 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:53.939957    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:54.004133 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:54.004091    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:54.097143 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:54.097105    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:54.189112 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:54.189080    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:54.290337 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:54.290307    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:54.387763 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:54.387730    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:55.046315 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:55.046253    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:55.124307 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:55.124274    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:55.461258 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:55.461219    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:55.513476 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:55.513442    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:58.504260 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:58.504220    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:14:58.582756 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:14:58.582717    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:01.126130 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:01.126091    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:15:01.126130 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:01.126128    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kdlkr\" (UID: \"b61b8094-e8b8-406b-89db-3c580535536d\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:15:01.126690 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:01.126245    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 02 13:15:01.126690 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:01.126256    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:15:01.126690 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:01.126263    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:15:01.126690 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:01.126304    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert podName:b61b8094-e8b8-406b-89db-3c580535536d nodeName:}" failed. No retries permitted until 2026-07-02 13:15:17.126286155 +0000 UTC m=+68.749079061 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-kdlkr" (UID: "b61b8094-e8b8-406b-89db-3c580535536d") : secret "networking-console-plugin-cert" not found
Jul 02 13:15:01.126690 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:01.126325    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls podName:0971a04b-36bc-46ca-9bce-762b11f50d31 nodeName:}" failed. No retries permitted until 2026-07-02 13:15:17.126315681 +0000 UTC m=+68.749108571 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls") pod "dns-default-h66tv" (UID: "0971a04b-36bc-46ca-9bce-762b11f50d31") : secret "dns-default-metrics-tls" not found
Jul 02 13:15:01.126690 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:01.126346    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 13:15:01.126690 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:01.126360    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f7c8df46f-cdx4k: secret "image-registry-tls" not found
Jul 02 13:15:01.126690 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:01.126421    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls podName:bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7 nodeName:}" failed. No retries permitted until 2026-07-02 13:15:17.126409491 +0000 UTC m=+68.749202389 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls") pod "image-registry-f7c8df46f-cdx4k" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7") : secret "image-registry-tls" not found
Jul 02 13:15:02.292496 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:02.292461    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:02.502265 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:02.502229    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:03.126054 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:03.126018    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:03.261455 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:03.261426    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:10.062305 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:10.062278    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-djnhg"
Jul 02 13:15:10.206863 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:10.206824    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:10.306792 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:10.306754    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:14.634038 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:14.633997    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs\") pod \"network-metrics-daemon-6nhcs\" (UID: \"aa2c8da0-b138-46ef-b9eb-10f6922697be\") " pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:15:14.636363 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:14.636344    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 02 13:15:14.645096 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:14.645070    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 02 13:15:14.645170 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:14.645142    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs podName:aa2c8da0-b138-46ef-b9eb-10f6922697be nodeName:}" failed. No retries permitted until 2026-07-02 13:16:18.645120463 +0000 UTC m=+130.267913361 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs") pod "network-metrics-daemon-6nhcs" (UID: "aa2c8da0-b138-46ef-b9eb-10f6922697be") : secret "metrics-daemon-secret" not found
Jul 02 13:15:14.734967 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:14.734931    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret\") pod \"global-pull-secret-syncer-bhrhl\" (UID: \"a8a9f309-636c-41da-a9a7-6d33ebced403\") " pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:15:14.735120 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:14.734988    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmxvp\" (UniqueName: \"kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp\") pod \"network-check-target-kv69m\" (UID: \"c41c0fa4-04c6-4e1b-9af2-0635451f3f26\") " pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:15:14.737274 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:14.737256    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 02 13:15:14.737397 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:14.737373    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 02 13:15:14.747473 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:14.747445    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 02 13:15:14.749151 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:14.749132    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8a9f309-636c-41da-a9a7-6d33ebced403-original-pull-secret\") pod \"global-pull-secret-syncer-bhrhl\" (UID: \"a8a9f309-636c-41da-a9a7-6d33ebced403\") " pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:15:14.758407 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:14.758363    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmxvp\" (UniqueName: \"kubernetes.io/projected/c41c0fa4-04c6-4e1b-9af2-0635451f3f26-kube-api-access-pmxvp\") pod \"network-check-target-kv69m\" (UID: \"c41c0fa4-04c6-4e1b-9af2-0635451f3f26\") " pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:15:15.001330 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:15.001293    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bhrhl"
Jul 02 13:15:15.014320 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:15.014296    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t5cpn\""
Jul 02 13:15:15.022638 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:15.022615    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:15:15.146815 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:15.146788    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bhrhl"]
Jul 02 13:15:15.150518 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:15:15.150489    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8a9f309_636c_41da_a9a7_6d33ebced403.slice/crio-2b7f2e02cd77874d3381354cb129872fe28d74d64646a1fc5de367f9f0549d8f WatchSource:0}: Error finding container 2b7f2e02cd77874d3381354cb129872fe28d74d64646a1fc5de367f9f0549d8f: Status 404 returned error can't find the container with id 2b7f2e02cd77874d3381354cb129872fe28d74d64646a1fc5de367f9f0549d8f
Jul 02 13:15:15.164192 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:15.164164    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kv69m"]
Jul 02 13:15:15.167512 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:15:15.167487    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc41c0fa4_04c6_4e1b_9af2_0635451f3f26.slice/crio-0329c588c9d654beb2e0e56974a82a479b76f692a76256724f645e074d283e7e WatchSource:0}: Error finding container 0329c588c9d654beb2e0e56974a82a479b76f692a76256724f645e074d283e7e: Status 404 returned error can't find the container with id 0329c588c9d654beb2e0e56974a82a479b76f692a76256724f645e074d283e7e
Jul 02 13:15:16.148716 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:16.148640    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kv69m" event={"ID":"c41c0fa4-04c6-4e1b-9af2-0635451f3f26","Type":"ContainerStarted","Data":"0329c588c9d654beb2e0e56974a82a479b76f692a76256724f645e074d283e7e"}
Jul 02 13:15:16.150158 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:16.150116    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bhrhl" event={"ID":"a8a9f309-636c-41da-a9a7-6d33ebced403","Type":"ContainerStarted","Data":"2b7f2e02cd77874d3381354cb129872fe28d74d64646a1fc5de367f9f0549d8f"}
Jul 02 13:15:16.522980 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:16.522944    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:16.623467 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:16.623363    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:17.156112 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:17.156074    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:15:17.156112 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:17.156111    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kdlkr\" (UID: \"b61b8094-e8b8-406b-89db-3c580535536d\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:15:17.156530 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:17.156140    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:15:17.156530 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:17.156234    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 02 13:15:17.156530 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:17.156303    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert podName:b61b8094-e8b8-406b-89db-3c580535536d nodeName:}" failed. No retries permitted until 2026-07-02 13:15:49.156286851 +0000 UTC m=+100.779079749 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-kdlkr" (UID: "b61b8094-e8b8-406b-89db-3c580535536d") : secret "networking-console-plugin-cert" not found
Jul 02 13:15:17.156530 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:17.156234    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 13:15:17.156530 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:17.156341    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f7c8df46f-cdx4k: secret "image-registry-tls" not found
Jul 02 13:15:17.156530 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:17.156380    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls podName:bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7 nodeName:}" failed. No retries permitted until 2026-07-02 13:15:49.156366883 +0000 UTC m=+100.779159784 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls") pod "image-registry-f7c8df46f-cdx4k" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7") : secret "image-registry-tls" not found
Jul 02 13:15:17.156530 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:17.156234    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:15:17.156530 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:17.156440    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls podName:0971a04b-36bc-46ca-9bce-762b11f50d31 nodeName:}" failed. No retries permitted until 2026-07-02 13:15:49.156429888 +0000 UTC m=+100.779222785 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls") pod "dns-default-h66tv" (UID: "0971a04b-36bc-46ca-9bce-762b11f50d31") : secret "dns-default-metrics-tls" not found
Jul 02 13:15:17.482152 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:17.482111    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:17.547426 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:17.547365    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:17.815912 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:17.815804    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:19.159289 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:19.159248    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kv69m" event={"ID":"c41c0fa4-04c6-4e1b-9af2-0635451f3f26","Type":"ContainerStarted","Data":"363987fb650361f40972337e496464b476c4dd2591a8058476592048639c019d"}
Jul 02 13:15:19.159754 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:19.159378    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:15:19.176549 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:19.176500    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kv69m" podStartSLOduration=66.994472975 podStartE2EDuration="1m10.176481637s" totalImagesPullingTime="3.182008662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:14:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:15:19.171884267 +0000 UTC m=+70.794677173" watchObservedRunningTime="2026-07-02 13:15:19.176481637 +0000 UTC m=+70.799274573"
Jul 02 13:15:20.440434 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:20.440349    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:21.165000 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:21.164918    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bhrhl" event={"ID":"a8a9f309-636c-41da-a9a7-6d33ebced403","Type":"ContainerStarted","Data":"4330c7df208120aaf9cd6044341d81a2e510e8523212198cdf6fad8f3beac707"}
Jul 02 13:15:21.180314 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:21.180267    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bhrhl" podStartSLOduration=65.666415442 podStartE2EDuration="1m11.180250054s" totalImagesPullingTime="5.513834612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:14:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:15:21.179913408 +0000 UTC m=+72.802706314" watchObservedRunningTime="2026-07-02 13:15:21.180250054 +0000 UTC m=+72.803042961"
Jul 02 13:15:23.438653 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:23.438615    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:23.489145 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:23.489112    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:30.245013 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:30.244975    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:34.736696 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:34.736658    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:35.162886 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:35.162782    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:49.196817 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:49.196775    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:15:49.196817 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:49.196819    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kdlkr\" (UID: \"b61b8094-e8b8-406b-89db-3c580535536d\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:15:49.197310 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:49.196861    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:15:49.197310 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:49.196935    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:15:49.197310 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:49.196979    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 13:15:49.197310 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:49.196986    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 02 13:15:49.197310 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:49.196994    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f7c8df46f-cdx4k: secret "image-registry-tls" not found
Jul 02 13:15:49.197310 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:49.197016    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls podName:0971a04b-36bc-46ca-9bce-762b11f50d31 nodeName:}" failed. No retries permitted until 2026-07-02 13:16:53.1969955 +0000 UTC m=+164.819788404 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls") pod "dns-default-h66tv" (UID: "0971a04b-36bc-46ca-9bce-762b11f50d31") : secret "dns-default-metrics-tls" not found
Jul 02 13:15:49.197310 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:49.197037    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert podName:b61b8094-e8b8-406b-89db-3c580535536d nodeName:}" failed. No retries permitted until 2026-07-02 13:16:53.197024954 +0000 UTC m=+164.819817853 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-kdlkr" (UID: "b61b8094-e8b8-406b-89db-3c580535536d") : secret "networking-console-plugin-cert" not found
Jul 02 13:15:49.197310 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:15:49.197054    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls podName:bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7 nodeName:}" failed. No retries permitted until 2026-07-02 13:16:53.197045623 +0000 UTC m=+164.819838511 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls") pod "image-registry-f7c8df46f-cdx4k" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7") : secret "image-registry-tls" not found
Jul 02 13:15:50.163757 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:50.163728    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kv69m"
Jul 02 13:15:55.518688 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:55.518647    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:15:55.619361 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:15:55.619323    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:16:18.701468 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:16:18.701433    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs\") pod \"network-metrics-daemon-6nhcs\" (UID: \"aa2c8da0-b138-46ef-b9eb-10f6922697be\") " pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:16:18.701960 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:16:18.701560    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 02 13:16:18.701960 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:16:18.701615    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs podName:aa2c8da0-b138-46ef-b9eb-10f6922697be nodeName:}" failed. No retries permitted until 2026-07-02 13:18:20.701600037 +0000 UTC m=+252.324392921 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs") pod "network-metrics-daemon-6nhcs" (UID: "aa2c8da0-b138-46ef-b9eb-10f6922697be") : secret "metrics-daemon-secret" not found
Jul 02 13:16:48.244824 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:16:48.244776    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k" podUID="bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7"
Jul 02 13:16:48.251305 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:16:48.251247    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr" podUID="b61b8094-e8b8-406b-89db-3c580535536d"
Jul 02 13:16:48.260355 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:16:48.260321    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-h66tv" podUID="0971a04b-36bc-46ca-9bce-762b11f50d31"
Jul 02 13:16:48.365579 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:16:48.365505    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h66tv"
Jul 02 13:16:48.365579 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:16:48.365519    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:16:48.365579 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:16:48.365528    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:16:48.907362 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:16:48.907328    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:16:53.266902 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:16:53.266866    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:16:53.266902 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:16:53.266905    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kdlkr\" (UID: \"b61b8094-e8b8-406b-89db-3c580535536d\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:16:53.267410 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:16:53.266948    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:16:53.267410 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:16:53.267020    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:16:53.267410 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:16:53.267035    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 02 13:16:53.267410 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:16:53.267070    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 13:16:53.267410 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:16:53.267082    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f7c8df46f-cdx4k: secret "image-registry-tls" not found
Jul 02 13:16:53.267410 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:16:53.267089    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert podName:b61b8094-e8b8-406b-89db-3c580535536d nodeName:}" failed. No retries permitted until 2026-07-02 13:18:55.267073145 +0000 UTC m=+286.889866032 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-kdlkr" (UID: "b61b8094-e8b8-406b-89db-3c580535536d") : secret "networking-console-plugin-cert" not found
Jul 02 13:16:53.267410 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:16:53.267104    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls podName:0971a04b-36bc-46ca-9bce-762b11f50d31 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:55.267096778 +0000 UTC m=+286.889889662 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls") pod "dns-default-h66tv" (UID: "0971a04b-36bc-46ca-9bce-762b11f50d31") : secret "dns-default-metrics-tls" not found
Jul 02 13:16:53.267410 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:16:53.267114    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls podName:bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:55.267108826 +0000 UTC m=+286.889901710 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls") pod "image-registry-f7c8df46f-cdx4k" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7") : secret "image-registry-tls" not found
Jul 02 13:16:59.892141 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:16:59.892097    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:17:22.901309 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:17:22.901226    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:02.043527 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:02.043489    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:20.800584 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:20.800544    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs\") pod \"network-metrics-daemon-6nhcs\" (UID: \"aa2c8da0-b138-46ef-b9eb-10f6922697be\") " pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:18:20.800958 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:20.800691    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 02 13:18:20.800958 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:20.800773    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs podName:aa2c8da0-b138-46ef-b9eb-10f6922697be nodeName:}" failed. No retries permitted until 2026-07-02 13:20:22.800757021 +0000 UTC m=+374.423549925 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs") pod "network-metrics-daemon-6nhcs" (UID: "aa2c8da0-b138-46ef-b9eb-10f6922697be") : secret "metrics-daemon-secret" not found
Jul 02 13:18:26.462295 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.462263    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-prr7j"]
Jul 02 13:18:26.465035 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.465018    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn"]
Jul 02 13:18:26.465227 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.465134    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-prr7j"
Jul 02 13:18:26.467971 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.467948    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"]
Jul 02 13:18:26.468123 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.468103    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn"
Jul 02 13:18:26.469442 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.468763    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-vpv9g\""
Jul 02 13:18:26.469442 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.469168    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 02 13:18:26.469442 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.469209    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 02 13:18:26.470691 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.470672    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"
Jul 02 13:18:26.472267 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.471218    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 02 13:18:26.472551 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.472534    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 02 13:18:26.472750 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.472727    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-twgmz\""
Jul 02 13:18:26.473421 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.473376    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 02 13:18:26.474447 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.473720    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 02 13:18:26.474447 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.473913    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wbqt7\""
Jul 02 13:18:26.475402 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.475248    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 02 13:18:26.475682 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.475595    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 02 13:18:26.475682 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.475673    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 02 13:18:26.476421 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.476403    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-prr7j"]
Jul 02 13:18:26.481827 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.481804    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn"]
Jul 02 13:18:26.491450 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.491427    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"]
Jul 02 13:18:26.542560 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.542527    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-z4gzj\" (UID: \"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"
Jul 02 13:18:26.542560 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.542561    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-z4gzj\" (UID: \"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"
Jul 02 13:18:26.542753 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.542579    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df5xg\" (UniqueName: \"kubernetes.io/projected/0604d44c-2c9d-4260-8edb-64b85f9c321c-kube-api-access-df5xg\") pod \"volume-data-source-validator-69d9f679db-prr7j\" (UID: \"0604d44c-2c9d-4260-8edb-64b85f9c321c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-prr7j"
Jul 02 13:18:26.542753 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.542682    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-5qjjn\" (UID: \"af4235e1-a082-492c-aebb-7ef694d20028\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn"
Jul 02 13:18:26.542753 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.542711    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78jvp\" (UniqueName: \"kubernetes.io/projected/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-kube-api-access-78jvp\") pod \"cluster-monitoring-operator-54cb84584-z4gzj\" (UID: \"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"
Jul 02 13:18:26.542753 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.542725    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbfbq\" (UniqueName: \"kubernetes.io/projected/af4235e1-a082-492c-aebb-7ef694d20028-kube-api-access-wbfbq\") pod \"cluster-samples-operator-9ddc5ff76-5qjjn\" (UID: \"af4235e1-a082-492c-aebb-7ef694d20028\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn"
Jul 02 13:18:26.559369 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.559335    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9"]
Jul 02 13:18:26.562288 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.562269    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-twcbv"]
Jul 02 13:18:26.562430 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.562399    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9"
Jul 02 13:18:26.564670 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.564651    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 02 13:18:26.564776 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.564728    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 02 13:18:26.564831 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.564789    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wdzhw\""
Jul 02 13:18:26.564969 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.564949    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5f67b55555-847dx"]
Jul 02 13:18:26.565079 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.564986    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 02 13:18:26.565079 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.565068    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-twcbv"
Jul 02 13:18:26.565216 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.565203    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 02 13:18:26.566481 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.566466    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-xgns8\""
Jul 02 13:18:26.567627 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.567611    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.569659 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.569643    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 02 13:18:26.569765 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.569675    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 02 13:18:26.569765 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.569719    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-twwjz\""
Jul 02 13:18:26.569765 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.569746    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 02 13:18:26.569974 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.569961    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 02 13:18:26.574122 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.574089    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9"]
Jul 02 13:18:26.575361 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.575344    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-twcbv"]
Jul 02 13:18:26.575725 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.575708    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 02 13:18:26.577603 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.577584    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5f67b55555-847dx"]
Jul 02 13:18:26.643647 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.643617    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrr47\" (UniqueName: \"kubernetes.io/projected/273bcb09-ea54-4145-af21-7392e86cdb9b-kube-api-access-xrr47\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.643831 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.643674    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4whhm\" (UniqueName: \"kubernetes.io/projected/29eede9f-c968-489b-82c6-12242f0b1801-kube-api-access-4whhm\") pod \"service-ca-operator-5d7b7dc95c-2xqx9\" (UID: \"29eede9f-c968-489b-82c6-12242f0b1801\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9"
Jul 02 13:18:26.643831 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.643702    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/29eede9f-c968-489b-82c6-12242f0b1801-config\") pod \"service-ca-operator-5d7b7dc95c-2xqx9\" (UID: \"29eede9f-c968-489b-82c6-12242f0b1801\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9"
Jul 02 13:18:26.643831 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.643718    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxbgc\" (UniqueName: \"kubernetes.io/projected/6b7d6fd3-387f-431c-88d4-43148130bac4-kube-api-access-hxbgc\") pod \"network-check-source-56499946bc-twcbv\" (UID: \"6b7d6fd3-387f-431c-88d4-43148130bac4\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-twcbv"
Jul 02 13:18:26.643831 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.643748    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-z4gzj\" (UID: \"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"
Jul 02 13:18:26.643831 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.643766    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-5qjjn\" (UID: \"af4235e1-a082-492c-aebb-7ef694d20028\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn"
Jul 02 13:18:26.643831 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.643781    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/273bcb09-ea54-4145-af21-7392e86cdb9b-snapshots\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.643831 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.643797    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/273bcb09-ea54-4145-af21-7392e86cdb9b-service-ca-bundle\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.644133 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:26.643892    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 02 13:18:26.644133 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.643897    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78jvp\" (UniqueName: \"kubernetes.io/projected/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-kube-api-access-78jvp\") pod \"cluster-monitoring-operator-54cb84584-z4gzj\" (UID: \"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"
Jul 02 13:18:26.644133 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.643953    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbfbq\" (UniqueName: \"kubernetes.io/projected/af4235e1-a082-492c-aebb-7ef694d20028-kube-api-access-wbfbq\") pod \"cluster-samples-operator-9ddc5ff76-5qjjn\" (UID: \"af4235e1-a082-492c-aebb-7ef694d20028\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn"
Jul 02 13:18:26.644133 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:26.643981    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls podName:af4235e1-a082-492c-aebb-7ef694d20028 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:27.14395972 +0000 UTC m=+258.766752620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-5qjjn" (UID: "af4235e1-a082-492c-aebb-7ef694d20028") : secret "samples-operator-tls" not found
Jul 02 13:18:26.644133 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.644014    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-z4gzj\" (UID: \"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"
Jul 02 13:18:26.644133 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.644045    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/273bcb09-ea54-4145-af21-7392e86cdb9b-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.644133 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.644088    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df5xg\" (UniqueName: \"kubernetes.io/projected/0604d44c-2c9d-4260-8edb-64b85f9c321c-kube-api-access-df5xg\") pod \"volume-data-source-validator-69d9f679db-prr7j\" (UID: \"0604d44c-2c9d-4260-8edb-64b85f9c321c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-prr7j"
Jul 02 13:18:26.644133 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:26.644102    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 02 13:18:26.644133 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.644115    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/273bcb09-ea54-4145-af21-7392e86cdb9b-serving-cert\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.644521 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.644148    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/273bcb09-ea54-4145-af21-7392e86cdb9b-tmp\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.644521 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:26.644163    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls podName:ed6ac0b5-82f0-4681-ab35-abe5a4a50d80 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:27.144148001 +0000 UTC m=+258.766940902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-z4gzj" (UID: "ed6ac0b5-82f0-4681-ab35-abe5a4a50d80") : secret "cluster-monitoring-operator-tls" not found
Jul 02 13:18:26.644521 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.644219    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/29eede9f-c968-489b-82c6-12242f0b1801-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-2xqx9\" (UID: \"29eede9f-c968-489b-82c6-12242f0b1801\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9"
Jul 02 13:18:26.644521 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.644446    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-z4gzj\" (UID: \"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"
Jul 02 13:18:26.653749 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.653725    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-df5xg\" (UniqueName: \"kubernetes.io/projected/0604d44c-2c9d-4260-8edb-64b85f9c321c-kube-api-access-df5xg\") pod \"volume-data-source-validator-69d9f679db-prr7j\" (UID: \"0604d44c-2c9d-4260-8edb-64b85f9c321c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-prr7j"
Jul 02 13:18:26.653941 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.653926    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78jvp\" (UniqueName: \"kubernetes.io/projected/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-kube-api-access-78jvp\") pod \"cluster-monitoring-operator-54cb84584-z4gzj\" (UID: \"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"
Jul 02 13:18:26.654311 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.654293    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbfbq\" (UniqueName: \"kubernetes.io/projected/af4235e1-a082-492c-aebb-7ef694d20028-kube-api-access-wbfbq\") pod \"cluster-samples-operator-9ddc5ff76-5qjjn\" (UID: \"af4235e1-a082-492c-aebb-7ef694d20028\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn"
Jul 02 13:18:26.726553 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.726453    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:26.745482 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.745457    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4whhm\" (UniqueName: \"kubernetes.io/projected/29eede9f-c968-489b-82c6-12242f0b1801-kube-api-access-4whhm\") pod \"service-ca-operator-5d7b7dc95c-2xqx9\" (UID: \"29eede9f-c968-489b-82c6-12242f0b1801\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9"
Jul 02 13:18:26.745555 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.745490    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/29eede9f-c968-489b-82c6-12242f0b1801-config\") pod \"service-ca-operator-5d7b7dc95c-2xqx9\" (UID: \"29eede9f-c968-489b-82c6-12242f0b1801\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9"
Jul 02 13:18:26.745555 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.745510    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxbgc\" (UniqueName: \"kubernetes.io/projected/6b7d6fd3-387f-431c-88d4-43148130bac4-kube-api-access-hxbgc\") pod \"network-check-source-56499946bc-twcbv\" (UID: \"6b7d6fd3-387f-431c-88d4-43148130bac4\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-twcbv"
Jul 02 13:18:26.745653 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.745563    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/273bcb09-ea54-4145-af21-7392e86cdb9b-snapshots\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.745653 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.745585    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/273bcb09-ea54-4145-af21-7392e86cdb9b-service-ca-bundle\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.745750 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.745669    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/273bcb09-ea54-4145-af21-7392e86cdb9b-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.745750 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.745697    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/273bcb09-ea54-4145-af21-7392e86cdb9b-serving-cert\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.745833 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.745806    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/273bcb09-ea54-4145-af21-7392e86cdb9b-tmp\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.745883 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.745849    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/29eede9f-c968-489b-82c6-12242f0b1801-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-2xqx9\" (UID: \"29eede9f-c968-489b-82c6-12242f0b1801\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9"
Jul 02 13:18:26.745931 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.745884    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrr47\" (UniqueName: \"kubernetes.io/projected/273bcb09-ea54-4145-af21-7392e86cdb9b-kube-api-access-xrr47\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.746231 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.746203    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/29eede9f-c968-489b-82c6-12242f0b1801-config\") pod \"service-ca-operator-5d7b7dc95c-2xqx9\" (UID: \"29eede9f-c968-489b-82c6-12242f0b1801\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9"
Jul 02 13:18:26.746380 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.746268    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/273bcb09-ea54-4145-af21-7392e86cdb9b-tmp\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.746380 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.746364    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/273bcb09-ea54-4145-af21-7392e86cdb9b-service-ca-bundle\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.746529 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.746365    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/273bcb09-ea54-4145-af21-7392e86cdb9b-snapshots\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.746679 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.746659    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/273bcb09-ea54-4145-af21-7392e86cdb9b-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.747960 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.747944    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/273bcb09-ea54-4145-af21-7392e86cdb9b-serving-cert\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.748238 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.748222    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/29eede9f-c968-489b-82c6-12242f0b1801-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-2xqx9\" (UID: \"29eede9f-c968-489b-82c6-12242f0b1801\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9"
Jul 02 13:18:26.752624 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.752591    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4whhm\" (UniqueName: \"kubernetes.io/projected/29eede9f-c968-489b-82c6-12242f0b1801-kube-api-access-4whhm\") pod \"service-ca-operator-5d7b7dc95c-2xqx9\" (UID: \"29eede9f-c968-489b-82c6-12242f0b1801\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9"
Jul 02 13:18:26.752737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.752684    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrr47\" (UniqueName: \"kubernetes.io/projected/273bcb09-ea54-4145-af21-7392e86cdb9b-kube-api-access-xrr47\") pod \"insights-operator-5f67b55555-847dx\" (UID: \"273bcb09-ea54-4145-af21-7392e86cdb9b\") " pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.752833 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.752816    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxbgc\" (UniqueName: \"kubernetes.io/projected/6b7d6fd3-387f-431c-88d4-43148130bac4-kube-api-access-hxbgc\") pod \"network-check-source-56499946bc-twcbv\" (UID: \"6b7d6fd3-387f-431c-88d4-43148130bac4\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-twcbv"
Jul 02 13:18:26.777690 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.777664    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-prr7j"
Jul 02 13:18:26.828638 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.828560    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:26.872763 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.872730    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9"
Jul 02 13:18:26.880918 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.880894    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-twcbv"
Jul 02 13:18:26.885837 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.885815    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5f67b55555-847dx"
Jul 02 13:18:26.901914 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:26.901880    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-prr7j"]
Jul 02 13:18:26.905956 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:18:26.905926    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0604d44c_2c9d_4260_8edb_64b85f9c321c.slice/crio-429562c7b958d8ed1419eba938aa546034d23f98b1b45ea792dd693b1c168811 WatchSource:0}: Error finding container 429562c7b958d8ed1419eba938aa546034d23f98b1b45ea792dd693b1c168811: Status 404 returned error can't find the container with id 429562c7b958d8ed1419eba938aa546034d23f98b1b45ea792dd693b1c168811
Jul 02 13:18:27.015708 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:27.015678    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-twcbv"]
Jul 02 13:18:27.017767 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:18:27.017738    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b7d6fd3_387f_431c_88d4_43148130bac4.slice/crio-c92f887ed72e69e7fde55695c579d51b5f5c598e24a71903c9cd97a2b546ac1a WatchSource:0}: Error finding container c92f887ed72e69e7fde55695c579d51b5f5c598e24a71903c9cd97a2b546ac1a: Status 404 returned error can't find the container with id c92f887ed72e69e7fde55695c579d51b5f5c598e24a71903c9cd97a2b546ac1a
Jul 02 13:18:27.033495 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:27.031825    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9"]
Jul 02 13:18:27.053654 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:27.053498    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5f67b55555-847dx"]
Jul 02 13:18:27.058049 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:18:27.058022    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod273bcb09_ea54_4145_af21_7392e86cdb9b.slice/crio-f7f605b94e7af4ddc0d3a21e656d3af60beb0865f4d340b228c2a83649f1f740 WatchSource:0}: Error finding container f7f605b94e7af4ddc0d3a21e656d3af60beb0865f4d340b228c2a83649f1f740: Status 404 returned error can't find the container with id f7f605b94e7af4ddc0d3a21e656d3af60beb0865f4d340b228c2a83649f1f740
Jul 02 13:18:27.150287 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:27.150248    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-z4gzj\" (UID: \"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"
Jul 02 13:18:27.150503 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:27.150340    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-5qjjn\" (UID: \"af4235e1-a082-492c-aebb-7ef694d20028\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn"
Jul 02 13:18:27.150503 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:27.150424    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 02 13:18:27.150503 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:27.150471    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 02 13:18:27.150668 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:27.150522    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls podName:ed6ac0b5-82f0-4681-ab35-abe5a4a50d80 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:28.150504549 +0000 UTC m=+259.773297433 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-z4gzj" (UID: "ed6ac0b5-82f0-4681-ab35-abe5a4a50d80") : secret "cluster-monitoring-operator-tls" not found
Jul 02 13:18:27.150668 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:27.150538    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls podName:af4235e1-a082-492c-aebb-7ef694d20028 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:28.150531056 +0000 UTC m=+259.773323940 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-5qjjn" (UID: "af4235e1-a082-492c-aebb-7ef694d20028") : secret "samples-operator-tls" not found
Jul 02 13:18:27.602684 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:27.602642    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-twcbv" event={"ID":"6b7d6fd3-387f-431c-88d4-43148130bac4","Type":"ContainerStarted","Data":"86ab6cedd0a98acbe4f05eab9faedac3831bb0909afed1424f9a9aa6a0d9e439"}
Jul 02 13:18:27.603137 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:27.602712    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-twcbv" event={"ID":"6b7d6fd3-387f-431c-88d4-43148130bac4","Type":"ContainerStarted","Data":"c92f887ed72e69e7fde55695c579d51b5f5c598e24a71903c9cd97a2b546ac1a"}
Jul 02 13:18:27.605115 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:27.605087    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9" event={"ID":"29eede9f-c968-489b-82c6-12242f0b1801","Type":"ContainerStarted","Data":"64c42e4ffb2e07ae8cf2daef45448964d575b4de1c4ccc342427bc4da129867a"}
Jul 02 13:18:27.606550 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:27.606504    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-847dx" event={"ID":"273bcb09-ea54-4145-af21-7392e86cdb9b","Type":"ContainerStarted","Data":"f7f605b94e7af4ddc0d3a21e656d3af60beb0865f4d340b228c2a83649f1f740"}
Jul 02 13:18:27.607953 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:27.607806    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-prr7j" event={"ID":"0604d44c-2c9d-4260-8edb-64b85f9c321c","Type":"ContainerStarted","Data":"429562c7b958d8ed1419eba938aa546034d23f98b1b45ea792dd693b1c168811"}
Jul 02 13:18:27.616148 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:27.616098    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-56499946bc-twcbv" podStartSLOduration=1.616078848 podStartE2EDuration="1.616078848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:18:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:18:27.61500104 +0000 UTC m=+259.237793946" watchObservedRunningTime="2026-07-02 13:18:27.616078848 +0000 UTC m=+259.238871734"
Jul 02 13:18:28.160907 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:28.160863    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-z4gzj\" (UID: \"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"
Jul 02 13:18:28.161100 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:28.161022    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 02 13:18:28.161100 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:28.161048    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-5qjjn\" (UID: \"af4235e1-a082-492c-aebb-7ef694d20028\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn"
Jul 02 13:18:28.161100 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:28.161088    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls podName:ed6ac0b5-82f0-4681-ab35-abe5a4a50d80 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:30.161067036 +0000 UTC m=+261.783859936 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-z4gzj" (UID: "ed6ac0b5-82f0-4681-ab35-abe5a4a50d80") : secret "cluster-monitoring-operator-tls" not found
Jul 02 13:18:28.161280 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:28.161148    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 02 13:18:28.161280 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:28.161200    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls podName:af4235e1-a082-492c-aebb-7ef694d20028 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:30.161184248 +0000 UTC m=+261.783977149 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-5qjjn" (UID: "af4235e1-a082-492c-aebb-7ef694d20028") : secret "samples-operator-tls" not found
Jul 02 13:18:28.613402 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:28.613354    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-prr7j" event={"ID":"0604d44c-2c9d-4260-8edb-64b85f9c321c","Type":"ContainerStarted","Data":"240d806baae7c0b2a4cc1695f002b0787d4032b9ea8bb0fab96b0b3d26f4643b"}
Jul 02 13:18:28.627037 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:28.626983    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-prr7j" podStartSLOduration=1.251690172 podStartE2EDuration="2.62696305s" totalImagesPullingTime="1.375272878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:18:28.625549574 +0000 UTC m=+260.248342480" watchObservedRunningTime="2026-07-02 13:18:28.62696305 +0000 UTC m=+260.249755957"
Jul 02 13:18:28.692085 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:28.692045    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:28.745327 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:28.745288    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:30.181218 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.181182    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-z4gzj\" (UID: \"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"
Jul 02 13:18:30.181713 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.181275    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-5qjjn\" (UID: \"af4235e1-a082-492c-aebb-7ef694d20028\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn"
Jul 02 13:18:30.181713 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:30.181338    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 02 13:18:30.181713 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:30.181372    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 02 13:18:30.181713 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:30.181412    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls podName:ed6ac0b5-82f0-4681-ab35-abe5a4a50d80 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:34.181397351 +0000 UTC m=+265.804190248 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-z4gzj" (UID: "ed6ac0b5-82f0-4681-ab35-abe5a4a50d80") : secret "cluster-monitoring-operator-tls" not found
Jul 02 13:18:30.181713 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:30.181427    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls podName:af4235e1-a082-492c-aebb-7ef694d20028 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:34.181421319 +0000 UTC m=+265.804214203 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-5qjjn" (UID: "af4235e1-a082-492c-aebb-7ef694d20028") : secret "samples-operator-tls" not found
Jul 02 13:18:30.620748 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.620711    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-847dx" event={"ID":"273bcb09-ea54-4145-af21-7392e86cdb9b","Type":"ContainerStarted","Data":"c7c4f96d94087ed2133b8f8083f5b1b41c9a97ec028e34818794c75e61b9b657"}
Jul 02 13:18:30.622365 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.622335    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9" event={"ID":"29eede9f-c968-489b-82c6-12242f0b1801","Type":"ContainerStarted","Data":"50c177681fde737df2b1a534b962d20432b6458c338c9994e96be2274dbfdd0d"}
Jul 02 13:18:30.635334 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.635294    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5f67b55555-847dx" podStartSLOduration=1.732476656 podStartE2EDuration="4.635279382s" totalImagesPullingTime="2.902802726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:18:30.634278634 +0000 UTC m=+262.257071533" watchObservedRunningTime="2026-07-02 13:18:30.635279382 +0000 UTC m=+262.258072288"
Jul 02 13:18:30.648594 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.648542    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9" podStartSLOduration=1.7191335030000001 podStartE2EDuration="4.64852662s" totalImagesPullingTime="2.929393117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:18:30.648082404 +0000 UTC m=+262.270875310" watchObservedRunningTime="2026-07-02 13:18:30.64852662 +0000 UTC m=+262.271319531"
Jul 02 13:18:30.775057 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.775020    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v8fdm"]
Jul 02 13:18:30.778505 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.778480    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:30.780862 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.780838    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 02 13:18:30.781316 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.781295    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 02 13:18:30.781450 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.781348    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rz6tf\""
Jul 02 13:18:30.798065 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.798042    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v8fdm"]
Jul 02 13:18:30.847357 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.847318    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:30.887449 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.887350    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:30.887449 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.887437    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:30.887621 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.887510    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnrhw\" (UniqueName: \"kubernetes.io/projected/30373957-8786-4661-8e15-0a26ec06fd66-kube-api-access-qnrhw\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:30.887621 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.887552    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/30373957-8786-4661-8e15-0a26ec06fd66-crio-socket\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:30.887621 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.887598    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/30373957-8786-4661-8e15-0a26ec06fd66-data-volume\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:30.897547 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.897516    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:30.987980 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.987944    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:30.988163 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.988011    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnrhw\" (UniqueName: \"kubernetes.io/projected/30373957-8786-4661-8e15-0a26ec06fd66-kube-api-access-qnrhw\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:30.988163 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.988057    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/30373957-8786-4661-8e15-0a26ec06fd66-crio-socket\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:30.988163 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:30.988062    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 13:18:30.988163 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.988101    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/30373957-8786-4661-8e15-0a26ec06fd66-data-volume\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:30.988163 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:30.988126    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm podName:30373957-8786-4661-8e15-0a26ec06fd66 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:31.488108813 +0000 UTC m=+263.110901701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v8fdm" (UID: "30373957-8786-4661-8e15-0a26ec06fd66") : configmap "kube-rbac-proxy" not found
Jul 02 13:18:30.988448 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.988198    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:30.988448 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.988206    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/30373957-8786-4661-8e15-0a26ec06fd66-crio-socket\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:30.988448 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:30.988295    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:18:30.988448 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:30.988332    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls podName:30373957-8786-4661-8e15-0a26ec06fd66 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:31.488320911 +0000 UTC m=+263.111113815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v8fdm" (UID: "30373957-8786-4661-8e15-0a26ec06fd66") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:18:30.988593 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.988454    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/30373957-8786-4661-8e15-0a26ec06fd66-data-volume\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:30.997693 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:30.997668    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnrhw\" (UniqueName: \"kubernetes.io/projected/30373957-8786-4661-8e15-0a26ec06fd66-kube-api-access-qnrhw\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:31.238411 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.238337    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:31.338096 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.338062    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:31.491819 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.491739    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:31.491971 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:31.491869    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 13:18:31.491971 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.491887    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:31.491971 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:31.491934    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm podName:30373957-8786-4661-8e15-0a26ec06fd66 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:32.491916378 +0000 UTC m=+264.114709284 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v8fdm" (UID: "30373957-8786-4661-8e15-0a26ec06fd66") : configmap "kube-rbac-proxy" not found
Jul 02 13:18:31.492085 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:31.491996    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:18:31.492085 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:31.492041    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls podName:30373957-8786-4661-8e15-0a26ec06fd66 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:32.492028464 +0000 UTC m=+264.114821365 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v8fdm" (UID: "30373957-8786-4661-8e15-0a26ec06fd66") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:18:31.655289 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.655251    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:31.684130 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.684099    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-5dkvq"]
Jul 02 13:18:31.687159 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.687138    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-5dkvq"
Jul 02 13:18:31.689035 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.689005    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 02 13:18:31.689193 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.689047    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5xx9f\""
Jul 02 13:18:31.689468 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.689451    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 02 13:18:31.695122 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.695101    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-5dkvq"]
Jul 02 13:18:31.717427 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.717376    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:31.773221 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.773137    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:31.793962 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.793929    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpf29\" (UniqueName: \"kubernetes.io/projected/a9972a0a-39f3-4dc1-b478-cbaa91d3a3b0-kube-api-access-qpf29\") pod \"migrator-786f9fbc5-5dkvq\" (UID: \"a9972a0a-39f3-4dc1-b478-cbaa91d3a3b0\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-5dkvq"
Jul 02 13:18:31.895038 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.895006    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpf29\" (UniqueName: \"kubernetes.io/projected/a9972a0a-39f3-4dc1-b478-cbaa91d3a3b0-kube-api-access-qpf29\") pod \"migrator-786f9fbc5-5dkvq\" (UID: \"a9972a0a-39f3-4dc1-b478-cbaa91d3a3b0\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-5dkvq"
Jul 02 13:18:31.908723 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.908692    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpf29\" (UniqueName: \"kubernetes.io/projected/a9972a0a-39f3-4dc1-b478-cbaa91d3a3b0-kube-api-access-qpf29\") pod \"migrator-786f9fbc5-5dkvq\" (UID: \"a9972a0a-39f3-4dc1-b478-cbaa91d3a3b0\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-5dkvq"
Jul 02 13:18:31.944295 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.944259    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:31.996608 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:31.996578    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-5dkvq"
Jul 02 13:18:32.043826 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:32.043796    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:32.113846 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:32.113815    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-5dkvq"]
Jul 02 13:18:32.116868 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:18:32.116839    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9972a0a_39f3_4dc1_b478_cbaa91d3a3b0.slice/crio-cf5f9cf71173667a75d90d0d2df50daaaa5e8b3369165123fe72a139613554cf WatchSource:0}: Error finding container cf5f9cf71173667a75d90d0d2df50daaaa5e8b3369165123fe72a139613554cf: Status 404 returned error can't find the container with id cf5f9cf71173667a75d90d0d2df50daaaa5e8b3369165123fe72a139613554cf
Jul 02 13:18:32.500703 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:32.500657    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:32.501094 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:32.500798    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:32.501094 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:32.500802    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 13:18:32.501094 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:32.500886    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm podName:30373957-8786-4661-8e15-0a26ec06fd66 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:34.500866598 +0000 UTC m=+266.123659486 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v8fdm" (UID: "30373957-8786-4661-8e15-0a26ec06fd66") : configmap "kube-rbac-proxy" not found
Jul 02 13:18:32.501094 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:32.500950    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:18:32.501094 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:32.501000    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls podName:30373957-8786-4661-8e15-0a26ec06fd66 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:34.5009878 +0000 UTC m=+266.123780700 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v8fdm" (UID: "30373957-8786-4661-8e15-0a26ec06fd66") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:18:32.630096 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:32.630056    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-5dkvq" event={"ID":"a9972a0a-39f3-4dc1-b478-cbaa91d3a3b0","Type":"ContainerStarted","Data":"cf5f9cf71173667a75d90d0d2df50daaaa5e8b3369165123fe72a139613554cf"}
Jul 02 13:18:32.773950 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:32.773813    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wfmns_fb05ec42-ba16-4d8b-82e1-1e689b75392d/dns-node-resolver/0.log"
Jul 02 13:18:33.542311 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.542278    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-nr579"]
Jul 02 13:18:33.545530 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.545507    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7bb4f4bcc7-nr579"
Jul 02 13:18:33.547347 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.547326    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 02 13:18:33.547468 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.547330    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 02 13:18:33.547661 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.547644    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-lf46l\""
Jul 02 13:18:33.547753 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.547656    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 02 13:18:33.547753 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.547674    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 02 13:18:33.554208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.554188    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-nr579"]
Jul 02 13:18:33.578811 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.578783    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:33.634571 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.634498    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-5dkvq" event={"ID":"a9972a0a-39f3-4dc1-b478-cbaa91d3a3b0","Type":"ContainerStarted","Data":"ece68c18c09d4b277d06b0ecb84ce7e5cd3b38b47561bd2149b8af919b62e2dc"}
Jul 02 13:18:33.634571 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.634547    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-5dkvq" event={"ID":"a9972a0a-39f3-4dc1-b478-cbaa91d3a3b0","Type":"ContainerStarted","Data":"f7a258314f86db325a4a44690682d9d43e36d9dec7399b7d664d9b9318738926"}
Jul 02 13:18:33.666962 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.666912    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-5dkvq" podStartSLOduration=1.447194546 podStartE2EDuration="2.666895761s" totalImagesPullingTime="1.219701215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:18:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:18:33.665047267 +0000 UTC m=+265.287840172" watchObservedRunningTime="2026-07-02 13:18:33.666895761 +0000 UTC m=+265.289688667"
Jul 02 13:18:33.710710 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.710682    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/73b869a7-1fdb-40f4-a427-717cfd4c1c6c-signing-key\") pod \"service-ca-7bb4f4bcc7-nr579\" (UID: \"73b869a7-1fdb-40f4-a427-717cfd4c1c6c\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-nr579"
Jul 02 13:18:33.710811 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.710715    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9frtl\" (UniqueName: \"kubernetes.io/projected/73b869a7-1fdb-40f4-a427-717cfd4c1c6c-kube-api-access-9frtl\") pod \"service-ca-7bb4f4bcc7-nr579\" (UID: \"73b869a7-1fdb-40f4-a427-717cfd4c1c6c\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-nr579"
Jul 02 13:18:33.710811 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.710801    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/73b869a7-1fdb-40f4-a427-717cfd4c1c6c-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-nr579\" (UID: \"73b869a7-1fdb-40f4-a427-717cfd4c1c6c\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-nr579"
Jul 02 13:18:33.773752 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.773728    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-q9xvb_cc015ede-26b1-49e2-9812-626d373e658f/node-ca/0.log"
Jul 02 13:18:33.811562 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.811523    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/73b869a7-1fdb-40f4-a427-717cfd4c1c6c-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-nr579\" (UID: \"73b869a7-1fdb-40f4-a427-717cfd4c1c6c\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-nr579"
Jul 02 13:18:33.811778 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.811759    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/73b869a7-1fdb-40f4-a427-717cfd4c1c6c-signing-key\") pod \"service-ca-7bb4f4bcc7-nr579\" (UID: \"73b869a7-1fdb-40f4-a427-717cfd4c1c6c\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-nr579"
Jul 02 13:18:33.811844 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.811795    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9frtl\" (UniqueName: \"kubernetes.io/projected/73b869a7-1fdb-40f4-a427-717cfd4c1c6c-kube-api-access-9frtl\") pod \"service-ca-7bb4f4bcc7-nr579\" (UID: \"73b869a7-1fdb-40f4-a427-717cfd4c1c6c\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-nr579"
Jul 02 13:18:33.812257 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.812236    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/73b869a7-1fdb-40f4-a427-717cfd4c1c6c-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-nr579\" (UID: \"73b869a7-1fdb-40f4-a427-717cfd4c1c6c\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-nr579"
Jul 02 13:18:33.814079 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.814056    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/73b869a7-1fdb-40f4-a427-717cfd4c1c6c-signing-key\") pod \"service-ca-7bb4f4bcc7-nr579\" (UID: \"73b869a7-1fdb-40f4-a427-717cfd4c1c6c\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-nr579"
Jul 02 13:18:33.819123 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.819095    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9frtl\" (UniqueName: \"kubernetes.io/projected/73b869a7-1fdb-40f4-a427-717cfd4c1c6c-kube-api-access-9frtl\") pod \"service-ca-7bb4f4bcc7-nr579\" (UID: \"73b869a7-1fdb-40f4-a427-717cfd4c1c6c\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-nr579"
Jul 02 13:18:33.853886 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.853852    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7bb4f4bcc7-nr579"
Jul 02 13:18:33.972262 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:33.972229    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-nr579"]
Jul 02 13:18:33.974855 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:18:33.974829    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73b869a7_1fdb_40f4_a427_717cfd4c1c6c.slice/crio-ce06f4d292b31c2332adc0c8d39bfb1e073c3760457c73934330b8914ed033b6 WatchSource:0}: Error finding container ce06f4d292b31c2332adc0c8d39bfb1e073c3760457c73934330b8914ed033b6: Status 404 returned error can't find the container with id ce06f4d292b31c2332adc0c8d39bfb1e073c3760457c73934330b8914ed033b6
Jul 02 13:18:34.215352 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:34.215316    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-5qjjn\" (UID: \"af4235e1-a082-492c-aebb-7ef694d20028\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn"
Jul 02 13:18:34.215554 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:34.215501    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-z4gzj\" (UID: \"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"
Jul 02 13:18:34.215554 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:34.215511    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 02 13:18:34.215666 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:34.215583    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls podName:af4235e1-a082-492c-aebb-7ef694d20028 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:42.215561798 +0000 UTC m=+273.838354701 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-5qjjn" (UID: "af4235e1-a082-492c-aebb-7ef694d20028") : secret "samples-operator-tls" not found
Jul 02 13:18:34.215666 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:34.215612    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 02 13:18:34.215666 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:34.215662    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls podName:ed6ac0b5-82f0-4681-ab35-abe5a4a50d80 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:42.215648268 +0000 UTC m=+273.838441167 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-z4gzj" (UID: "ed6ac0b5-82f0-4681-ab35-abe5a4a50d80") : secret "cluster-monitoring-operator-tls" not found
Jul 02 13:18:34.518021 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:34.517928    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:34.518021 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:34.517988    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:34.518201 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:34.518081    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:18:34.518201 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:34.518095    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 13:18:34.518201 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:34.518146    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm podName:30373957-8786-4661-8e15-0a26ec06fd66 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:38.518132223 +0000 UTC m=+270.140925113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v8fdm" (UID: "30373957-8786-4661-8e15-0a26ec06fd66") : configmap "kube-rbac-proxy" not found
Jul 02 13:18:34.518201 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:34.518163    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls podName:30373957-8786-4661-8e15-0a26ec06fd66 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:38.518155297 +0000 UTC m=+270.140948180 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v8fdm" (UID: "30373957-8786-4661-8e15-0a26ec06fd66") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:18:34.574604 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:34.574575    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dfhv6_6b13232c-9bcc-423b-a85f-1ef580a1d477/serve-healthcheck-canary/0.log"
Jul 02 13:18:34.639607 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:34.639572    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7bb4f4bcc7-nr579" event={"ID":"73b869a7-1fdb-40f4-a427-717cfd4c1c6c","Type":"ContainerStarted","Data":"059b0856013f8d1e62021e0dc651f189052396668ac18ce8c5b0b4a4094c6bb8"}
Jul 02 13:18:34.639792 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:34.639625    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7bb4f4bcc7-nr579" event={"ID":"73b869a7-1fdb-40f4-a427-717cfd4c1c6c","Type":"ContainerStarted","Data":"ce06f4d292b31c2332adc0c8d39bfb1e073c3760457c73934330b8914ed033b6"}
Jul 02 13:18:34.654848 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:34.654786    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7bb4f4bcc7-nr579" podStartSLOduration=1.654767747 podStartE2EDuration="1.654767747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:18:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:18:34.653153726 +0000 UTC m=+266.275946632" watchObservedRunningTime="2026-07-02 13:18:34.654767747 +0000 UTC m=+266.277560657"
Jul 02 13:18:35.458447 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:35.458411    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:35.823954 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:35.823856    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:36.423073 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:36.423034    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:37.022809 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:37.022526    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:37.226810 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:37.226775    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:37.555959 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:37.555923    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:37.658472 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:37.658425    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:37.832824 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:37.832732    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:38.224534 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:38.224505    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:38.554650 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:38.554567    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:38.554650 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:38.554634    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:38.554855 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:38.554732    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:18:38.554855 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:38.554816    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls podName:30373957-8786-4661-8e15-0a26ec06fd66 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:46.55479563 +0000 UTC m=+278.177588530 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v8fdm" (UID: "30373957-8786-4661-8e15-0a26ec06fd66") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:18:38.554855 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:38.554740    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 13:18:38.554982 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:38.554872    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm podName:30373957-8786-4661-8e15-0a26ec06fd66 nodeName:}" failed. No retries permitted until 2026-07-02 13:18:46.554857988 +0000 UTC m=+278.177650894 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v8fdm" (UID: "30373957-8786-4661-8e15-0a26ec06fd66") : configmap "kube-rbac-proxy" not found
Jul 02 13:18:38.828937 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:38.828852    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:38.928767 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:38.928733    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:39.225335 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:39.225300    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:39.424654 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:39.424615    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:40.027214 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:40.027173    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:40.669776 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:40.669740    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:40.820443 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:40.820407    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:40.926412 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:40.926292    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:41.054263 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:41.054218    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:41.318567 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:41.318530    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:41.429765 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:41.429730    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:42.239402 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:42.239363    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:42.286739 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:42.286704    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-z4gzj\" (UID: \"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"
Jul 02 13:18:42.286899 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:42.286827    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-5qjjn\" (UID: \"af4235e1-a082-492c-aebb-7ef694d20028\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn"
Jul 02 13:18:42.289242 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:42.289215    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ed6ac0b5-82f0-4681-ab35-abe5a4a50d80-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-z4gzj\" (UID: \"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"
Jul 02 13:18:42.289346 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:42.289275    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/af4235e1-a082-492c-aebb-7ef694d20028-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-5qjjn\" (UID: \"af4235e1-a082-492c-aebb-7ef694d20028\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn"
Jul 02 13:18:42.385446 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:42.385410    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn"
Jul 02 13:18:42.390451 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:42.390408    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"
Jul 02 13:18:42.425207 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:42.425169    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:42.531964 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:42.531940    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn"]
Jul 02 13:18:42.544831 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:42.544809    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj"]
Jul 02 13:18:42.562485 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:18:42.562453    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded6ac0b5_82f0_4681_ab35_abe5a4a50d80.slice/crio-fa5adfabff7efc3bdf6c2a0ed147292c75db1df85e53a6f58cd8c3550f12b057 WatchSource:0}: Error finding container fa5adfabff7efc3bdf6c2a0ed147292c75db1df85e53a6f58cd8c3550f12b057: Status 404 returned error can't find the container with id fa5adfabff7efc3bdf6c2a0ed147292c75db1df85e53a6f58cd8c3550f12b057
Jul 02 13:18:42.667789 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:42.667748    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn" event={"ID":"af4235e1-a082-492c-aebb-7ef694d20028","Type":"ContainerStarted","Data":"a938fcd84e7a1208ad274a7d0aa758441f98828bb6a04bb6550472f4354ad612"}
Jul 02 13:18:42.668951 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:42.668931    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj" event={"ID":"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80","Type":"ContainerStarted","Data":"fa5adfabff7efc3bdf6c2a0ed147292c75db1df85e53a6f58cd8c3550f12b057"}
Jul 02 13:18:42.827182 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:42.827089    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:43.027433 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:43.027396    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:43.424985 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:43.424948    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:43.991673 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:43.991581    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:44.046450 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:44.046408    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:44.426347 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:44.426246    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:44.670809 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:44.670770    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:44.821172 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:44.821140    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:45.470634 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:45.470597    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:45.570351 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:45.570320    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:45.671059 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:45.670997    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:45.683645 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:45.683612    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj" event={"ID":"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80","Type":"ContainerStarted","Data":"971e8608f5fe57edbebb6fe8d5e17786fdf273c2fed22e3af6cbcfc4cbf11b0d"}
Jul 02 13:18:45.685353 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:45.685327    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn" event={"ID":"af4235e1-a082-492c-aebb-7ef694d20028","Type":"ContainerStarted","Data":"3d270a81574907a32aa54737655f3e948e4e908ae86b0f5b3f39a063204e63f7"}
Jul 02 13:18:45.685474 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:45.685378    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn" event={"ID":"af4235e1-a082-492c-aebb-7ef694d20028","Type":"ContainerStarted","Data":"fa55fc6b9f71732a748c9738bcc2c137b15470a72ed759e3e4f76caa747b4b67"}
Jul 02 13:18:45.702792 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:45.702750    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj" podStartSLOduration=17.100071671 podStartE2EDuration="19.702736043s" totalImagesPullingTime="2.602664372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:18:45.702154226 +0000 UTC m=+277.324947135" watchObservedRunningTime="2026-07-02 13:18:45.702736043 +0000 UTC m=+277.325528949"
Jul 02 13:18:45.716284 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:45.716245    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-5qjjn" podStartSLOduration=17.132822708 podStartE2EDuration="19.716231861s" totalImagesPullingTime="2.583409153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:18:45.715177865 +0000 UTC m=+277.337970772" watchObservedRunningTime="2026-07-02 13:18:45.716231861 +0000 UTC m=+277.339024766"
Jul 02 13:18:45.729937 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:45.729849    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:46.043300 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:46.043208    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:46.628153 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:46.628117    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:46.628647 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:46.628172    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:46.628819 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:46.628797    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/30373957-8786-4661-8e15-0a26ec06fd66-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:46.630551 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:46.630529    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/30373957-8786-4661-8e15-0a26ec06fd66-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8fdm\" (UID: \"30373957-8786-4661-8e15-0a26ec06fd66\") " pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:46.692535 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:46.692503    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v8fdm"
Jul 02 13:18:46.828463 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:46.828420    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v8fdm"]
Jul 02 13:18:46.831351 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:18:46.831323    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30373957_8786_4661_8e15_0a26ec06fd66.slice/crio-6d3f42e0250152fca1480bd4e6d573a002e96d293a7f310298d5ae0f23c3007a WatchSource:0}: Error finding container 6d3f42e0250152fca1480bd4e6d573a002e96d293a7f310298d5ae0f23c3007a: Status 404 returned error can't find the container with id 6d3f42e0250152fca1480bd4e6d573a002e96d293a7f310298d5ae0f23c3007a
Jul 02 13:18:47.699682 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:47.699643    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v8fdm" event={"ID":"30373957-8786-4661-8e15-0a26ec06fd66","Type":"ContainerStarted","Data":"6d3f42e0250152fca1480bd4e6d573a002e96d293a7f310298d5ae0f23c3007a"}
Jul 02 13:18:51.299518 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:51.299465    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:51.366364 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:51.366322    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-h66tv" podUID="0971a04b-36bc-46ca-9bce-762b11f50d31"
Jul 02 13:18:51.366364 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:51.366334    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr" podUID="b61b8094-e8b8-406b-89db-3c580535536d"
Jul 02 13:18:51.366578 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:51.366349    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k" podUID="bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7"
Jul 02 13:18:51.399925 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:51.399891    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:51.544717 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:51.544677    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:51.715206 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:51.715128    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:18:51.715368 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:51.715155    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h66tv"
Jul 02 13:18:51.715368 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:51.715161    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:18:51.715517 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:51.715156    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v8fdm" event={"ID":"30373957-8786-4661-8e15-0a26ec06fd66","Type":"ContainerStarted","Data":"9b272751f5ac93fc02dff0c322f91b396ab303732224fe176070b4a6345bf2a2"}
Jul 02 13:18:51.910858 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:51.910823    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:52.153197 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:52.153077    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:52.319233 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:52.319197    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:52.536150 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:52.536113    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:52.720932 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:52.720890    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v8fdm" event={"ID":"30373957-8786-4661-8e15-0a26ec06fd66","Type":"ContainerStarted","Data":"23314be3d5c97bccc1b2592baee1af2273bfdbdd2e33aa5af8fa30bf2bd36bac"}
Jul 02 13:18:52.761695 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:52.761651    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:52.972899 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:52.972847    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:53.133286 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:53.133248    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:53.234016 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:53.233931    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:54.728943 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:54.728909    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v8fdm" event={"ID":"30373957-8786-4661-8e15-0a26ec06fd66","Type":"ContainerStarted","Data":"2437b1a743587d26363713cf94165c2542d2063a1aca27d06a5e2d1f2e61e098"}
Jul 02 13:18:54.748913 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:54.748861    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v8fdm" podStartSLOduration=17.365943544 podStartE2EDuration="24.748844298s" totalImagesPullingTime="7.382900754s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:18:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:18:54.748520093 +0000 UTC m=+286.371312998" watchObservedRunningTime="2026-07-02 13:18:54.748844298 +0000 UTC m=+286.371637204"
Jul 02 13:18:54.777577 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:54.777546    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:54.829306 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:54.829278    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:55.121173 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.121097    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-jxkcp"]
Jul 02 13:18:55.124314 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.124291    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-jxkcp"
Jul 02 13:18:55.126776 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.126752    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-fzmp9\""
Jul 02 13:18:55.126966 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.126938    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 02 13:18:55.133990 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.133960    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-jxkcp"]
Jul 02 13:18:55.202363 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.202328    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d87a4a62-6245-4a66-a3d2-ba1f70555c1d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-jxkcp\" (UID: \"d87a4a62-6245-4a66-a3d2-ba1f70555c1d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-jxkcp"
Jul 02 13:18:55.279570 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.279531    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:55.303254 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.303219    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:18:55.303454 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.303287    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d87a4a62-6245-4a66-a3d2-ba1f70555c1d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-jxkcp\" (UID: \"d87a4a62-6245-4a66-a3d2-ba1f70555c1d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-jxkcp"
Jul 02 13:18:55.303454 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.303328    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:18:55.303454 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.303346    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kdlkr\" (UID: \"b61b8094-e8b8-406b-89db-3c580535536d\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:18:55.305744 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.305709    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0971a04b-36bc-46ca-9bce-762b11f50d31-metrics-tls\") pod \"dns-default-h66tv\" (UID: \"0971a04b-36bc-46ca-9bce-762b11f50d31\") " pod="openshift-dns/dns-default-h66tv"
Jul 02 13:18:55.305909 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.305891    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b61b8094-e8b8-406b-89db-3c580535536d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-kdlkr\" (UID: \"b61b8094-e8b8-406b-89db-3c580535536d\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:18:55.306139 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.306118    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls\") pod \"image-registry-f7c8df46f-cdx4k\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") " pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:18:55.306212 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.306145    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d87a4a62-6245-4a66-a3d2-ba1f70555c1d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-jxkcp\" (UID: \"d87a4a62-6245-4a66-a3d2-ba1f70555c1d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-jxkcp"
Jul 02 13:18:55.318213 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.318189    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d2c87\""
Jul 02 13:18:55.318408 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.318363    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-vs9mx\""
Jul 02 13:18:55.318408 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.318370    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wn2v6\""
Jul 02 13:18:55.326864 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.326843    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:18:55.326989 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.326975    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"
Jul 02 13:18:55.327026 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.327007    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h66tv"
Jul 02 13:18:55.356414 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.356350    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:55.434752 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.434723    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-jxkcp"
Jul 02 13:18:55.495882 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.495776    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f7c8df46f-cdx4k"]
Jul 02 13:18:55.499146 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:18:55.499106    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb9e608_ff8c_4afc_8e6d_36d1b416b4c7.slice/crio-a94fa8cc509649d5fee5123f9943de818dfa428f24cd5508626f304b0c7dd845 WatchSource:0}: Error finding container a94fa8cc509649d5fee5123f9943de818dfa428f24cd5508626f304b0c7dd845: Status 404 returned error can't find the container with id a94fa8cc509649d5fee5123f9943de818dfa428f24cd5508626f304b0c7dd845
Jul 02 13:18:55.576063 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.576027    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-jxkcp"]
Jul 02 13:18:55.580819 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:18:55.580793    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd87a4a62_6245_4a66_a3d2_ba1f70555c1d.slice/crio-7f84513d6122e4c7acd7e56d4e16bc0e27cabcef3aeac5fffbbac69fa103efff WatchSource:0}: Error finding container 7f84513d6122e4c7acd7e56d4e16bc0e27cabcef3aeac5fffbbac69fa103efff: Status 404 returned error can't find the container with id 7f84513d6122e4c7acd7e56d4e16bc0e27cabcef3aeac5fffbbac69fa103efff
Jul 02 13:18:55.707315 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.707289    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr"]
Jul 02 13:18:55.710302 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.710268    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h66tv"]
Jul 02 13:18:55.710459 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:18:55.710347    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb61b8094_e8b8_406b_89db_3c580535536d.slice/crio-53047cd06894e1b756ff9b915a2bb31e17a1d06cceda28181a995b67e60e3871 WatchSource:0}: Error finding container 53047cd06894e1b756ff9b915a2bb31e17a1d06cceda28181a995b67e60e3871: Status 404 returned error can't find the container with id 53047cd06894e1b756ff9b915a2bb31e17a1d06cceda28181a995b67e60e3871
Jul 02 13:18:55.713622 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:18:55.713597    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0971a04b_36bc_46ca_9bce_762b11f50d31.slice/crio-f92cdab1f98a795c972cdde5b8d942de9ed8fcfff9e5a3fa939b48b0995b3254 WatchSource:0}: Error finding container f92cdab1f98a795c972cdde5b8d942de9ed8fcfff9e5a3fa939b48b0995b3254: Status 404 returned error can't find the container with id f92cdab1f98a795c972cdde5b8d942de9ed8fcfff9e5a3fa939b48b0995b3254
Jul 02 13:18:55.733918 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.733872    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h66tv" event={"ID":"0971a04b-36bc-46ca-9bce-762b11f50d31","Type":"ContainerStarted","Data":"f92cdab1f98a795c972cdde5b8d942de9ed8fcfff9e5a3fa939b48b0995b3254"}
Jul 02 13:18:55.735063 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.735041    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr" event={"ID":"b61b8094-e8b8-406b-89db-3c580535536d","Type":"ContainerStarted","Data":"53047cd06894e1b756ff9b915a2bb31e17a1d06cceda28181a995b67e60e3871"}
Jul 02 13:18:55.736078 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.736053    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-jxkcp" event={"ID":"d87a4a62-6245-4a66-a3d2-ba1f70555c1d","Type":"ContainerStarted","Data":"7f84513d6122e4c7acd7e56d4e16bc0e27cabcef3aeac5fffbbac69fa103efff"}
Jul 02 13:18:55.737477 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.737455    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k" event={"ID":"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7","Type":"ContainerStarted","Data":"097980bb9773183b525fe9d2498b0397250dde5d9786b7a002863a80c533ba06"}
Jul 02 13:18:55.737572 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.737494    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k" event={"ID":"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7","Type":"ContainerStarted","Data":"a94fa8cc509649d5fee5123f9943de818dfa428f24cd5508626f304b0c7dd845"}
Jul 02 13:18:55.737618 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.737591    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:18:55.755954 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:55.755896    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k" podStartSLOduration=285.75587348 podStartE2EDuration="4m45.75587348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:14:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:18:55.754249941 +0000 UTC m=+287.377042848" watchObservedRunningTime="2026-07-02 13:18:55.75587348 +0000 UTC m=+287.378666388"
Jul 02 13:18:58.748832 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:58.748786    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-jxkcp" event={"ID":"d87a4a62-6245-4a66-a3d2-ba1f70555c1d","Type":"ContainerStarted","Data":"3afd64d6765a4cb52fef88d5f4002e79c5d33b9056d8dd02ff68f970e5e531bf"}
Jul 02 13:18:58.749284 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:58.748962    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-jxkcp"
Jul 02 13:18:58.750589 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:58.750561    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h66tv" event={"ID":"0971a04b-36bc-46ca-9bce-762b11f50d31","Type":"ContainerStarted","Data":"d6310fe898ee8e6ff954197bf09ec1d1687040e853b4ba2e7414d27f3fbc297f"}
Jul 02 13:18:58.750727 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:58.750610    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h66tv" event={"ID":"0971a04b-36bc-46ca-9bce-762b11f50d31","Type":"ContainerStarted","Data":"660f4537f1e62734b3108da5200b5f8aafe6afa0e1d72a0f1d948ca0a15530a7"}
Jul 02 13:18:58.750727 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:58.750640    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-h66tv"
Jul 02 13:18:58.752121 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:58.752098    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr" event={"ID":"b61b8094-e8b8-406b-89db-3c580535536d","Type":"ContainerStarted","Data":"1786fed83b07f4eb748e13cede5228d416333a9ce4817d6e7a9f7ee241a15736"}
Jul 02 13:18:58.754200 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:58.754182    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-jxkcp"
Jul 02 13:18:58.762603 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:58.762560    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-jxkcp" podStartSLOduration=1.253567083 podStartE2EDuration="3.762546337s" totalImagesPullingTime="2.508979254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:18:58.761491223 +0000 UTC m=+290.384284128" watchObservedRunningTime="2026-07-02 13:18:58.762546337 +0000 UTC m=+290.385339242"
Jul 02 13:18:58.787757 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:58.787704    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-h66tv" podStartSLOduration=251.41166618 podStartE2EDuration="4m13.787686267s" totalImagesPullingTime="2.376020087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:18:58.786206369 +0000 UTC m=+290.408999276" watchObservedRunningTime="2026-07-02 13:18:58.787686267 +0000 UTC m=+290.410479172"
Jul 02 13:18:58.800089 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:58.800037    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-kdlkr" podStartSLOduration=267.419483725 podStartE2EDuration="4m29.800018023s" totalImagesPullingTime="2.380534298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:18:58.798958304 +0000 UTC m=+290.421751235" watchObservedRunningTime="2026-07-02 13:18:58.800018023 +0000 UTC m=+290.422810929"
Jul 02 13:18:58.882697 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:58.882663    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:58.934342 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:58.934283    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:58.983292 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:58.983258    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:59.675877 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.675843    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:59.716486 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.716454    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"]
Jul 02 13:18:59.719690 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.719673    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"
Jul 02 13:18:59.721950 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.721931    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 02 13:18:59.722065 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.721946    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-s6rqw\""
Jul 02 13:18:59.722065 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.721954    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 02 13:18:59.722065 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.722003    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 02 13:18:59.727490 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.727449    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"]
Jul 02 13:18:59.744960 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.744928    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:18:59.847680 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.847643    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbdsl\" (UniqueName: \"kubernetes.io/projected/9ae30b20-eccb-40a4-a74c-4ad792f93d8a-kube-api-access-zbdsl\") pod \"prometheus-operator-7b7d896497-qzk2p\" (UID: \"9ae30b20-eccb-40a4-a74c-4ad792f93d8a\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"
Jul 02 13:18:59.848143 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.847780    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ae30b20-eccb-40a4-a74c-4ad792f93d8a-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-qzk2p\" (UID: \"9ae30b20-eccb-40a4-a74c-4ad792f93d8a\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"
Jul 02 13:18:59.848143 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.847911    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ae30b20-eccb-40a4-a74c-4ad792f93d8a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-qzk2p\" (UID: \"9ae30b20-eccb-40a4-a74c-4ad792f93d8a\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"
Jul 02 13:18:59.848143 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.847947    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ae30b20-eccb-40a4-a74c-4ad792f93d8a-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-qzk2p\" (UID: \"9ae30b20-eccb-40a4-a74c-4ad792f93d8a\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"
Jul 02 13:18:59.948918 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.948879    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ae30b20-eccb-40a4-a74c-4ad792f93d8a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-qzk2p\" (UID: \"9ae30b20-eccb-40a4-a74c-4ad792f93d8a\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"
Jul 02 13:18:59.949094 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.948925    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ae30b20-eccb-40a4-a74c-4ad792f93d8a-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-qzk2p\" (UID: \"9ae30b20-eccb-40a4-a74c-4ad792f93d8a\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"
Jul 02 13:18:59.949094 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.948967    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbdsl\" (UniqueName: \"kubernetes.io/projected/9ae30b20-eccb-40a4-a74c-4ad792f93d8a-kube-api-access-zbdsl\") pod \"prometheus-operator-7b7d896497-qzk2p\" (UID: \"9ae30b20-eccb-40a4-a74c-4ad792f93d8a\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"
Jul 02 13:18:59.949225 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:59.949097    2570 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 02 13:18:59.949225 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:18:59.949158    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ae30b20-eccb-40a4-a74c-4ad792f93d8a-prometheus-operator-tls podName:9ae30b20-eccb-40a4-a74c-4ad792f93d8a nodeName:}" failed. No retries permitted until 2026-07-02 13:19:00.449142274 +0000 UTC m=+292.071935161 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/9ae30b20-eccb-40a4-a74c-4ad792f93d8a-prometheus-operator-tls") pod "prometheus-operator-7b7d896497-qzk2p" (UID: "9ae30b20-eccb-40a4-a74c-4ad792f93d8a") : secret "prometheus-operator-tls" not found
Jul 02 13:18:59.949225 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.949122    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ae30b20-eccb-40a4-a74c-4ad792f93d8a-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-qzk2p\" (UID: \"9ae30b20-eccb-40a4-a74c-4ad792f93d8a\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"
Jul 02 13:18:59.949795 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.949776    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ae30b20-eccb-40a4-a74c-4ad792f93d8a-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-qzk2p\" (UID: \"9ae30b20-eccb-40a4-a74c-4ad792f93d8a\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"
Jul 02 13:18:59.951332 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.951315    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ae30b20-eccb-40a4-a74c-4ad792f93d8a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-qzk2p\" (UID: \"9ae30b20-eccb-40a4-a74c-4ad792f93d8a\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"
Jul 02 13:18:59.960429 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:18:59.960380    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbdsl\" (UniqueName: \"kubernetes.io/projected/9ae30b20-eccb-40a4-a74c-4ad792f93d8a-kube-api-access-zbdsl\") pod \"prometheus-operator-7b7d896497-qzk2p\" (UID: \"9ae30b20-eccb-40a4-a74c-4ad792f93d8a\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"
Jul 02 13:19:00.075601 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:00.075561    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:00.453309 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:00.453282    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ae30b20-eccb-40a4-a74c-4ad792f93d8a-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-qzk2p\" (UID: \"9ae30b20-eccb-40a4-a74c-4ad792f93d8a\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"
Jul 02 13:19:00.455713 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:00.455682    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ae30b20-eccb-40a4-a74c-4ad792f93d8a-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-qzk2p\" (UID: \"9ae30b20-eccb-40a4-a74c-4ad792f93d8a\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"
Jul 02 13:19:00.629781 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:00.629743    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"
Jul 02 13:19:00.755782 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:00.755748    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-qzk2p"]
Jul 02 13:19:00.758737 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:19:00.758710    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ae30b20_eccb_40a4_a74c_4ad792f93d8a.slice/crio-df8826b21cfeff06070d7727e446c9d140b985d434a5798fcba2152bc3e00237 WatchSource:0}: Error finding container df8826b21cfeff06070d7727e446c9d140b985d434a5798fcba2152bc3e00237: Status 404 returned error can't find the container with id df8826b21cfeff06070d7727e446c9d140b985d434a5798fcba2152bc3e00237
Jul 02 13:19:01.763617 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:01.763582    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p" event={"ID":"9ae30b20-eccb-40a4-a74c-4ad792f93d8a","Type":"ContainerStarted","Data":"df8826b21cfeff06070d7727e446c9d140b985d434a5798fcba2152bc3e00237"}
Jul 02 13:19:02.657470 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:02.657367    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:02.768489 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:02.768451    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p" event={"ID":"9ae30b20-eccb-40a4-a74c-4ad792f93d8a","Type":"ContainerStarted","Data":"eb27d7229fc5dc38438b03f05c0127b11ab55e68a864449f5efd0a6b285949eb"}
Jul 02 13:19:02.769002 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:02.768512    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p" event={"ID":"9ae30b20-eccb-40a4-a74c-4ad792f93d8a","Type":"ContainerStarted","Data":"4fe39f57a7b23f36f84116f5a1dc41fff4d3adb9ffdbf84f7ccc26be086bf884"}
Jul 02 13:19:02.783018 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:02.782973    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7b7d896497-qzk2p" podStartSLOduration=2.159759136 podStartE2EDuration="3.782957346s" totalImagesPullingTime="1.62319821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:18:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:19:02.782161119 +0000 UTC m=+294.404954044" watchObservedRunningTime="2026-07-02 13:19:02.782957346 +0000 UTC m=+294.405750251"
Jul 02 13:19:02.893006 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:19:02.892964    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6nhcs" podUID="aa2c8da0-b138-46ef-b9eb-10f6922697be"
Jul 02 13:19:04.822740 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:04.822708    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:05.102861 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:05.102781    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:05.780633 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:05.780607    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:19:05.780785 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:05.780645    2570 generic.go:356] "Generic (PLEG): container finished" podID="ed6ac0b5-82f0-4681-ab35-abe5a4a50d80" containerID="971e8608f5fe57edbebb6fe8d5e17786fdf273c2fed22e3af6cbcfc4cbf11b0d" exitCode=2
Jul 02 13:19:05.780785 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:05.780685    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj" event={"ID":"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80","Type":"ContainerDied","Data":"971e8608f5fe57edbebb6fe8d5e17786fdf273c2fed22e3af6cbcfc4cbf11b0d"}
Jul 02 13:19:05.780996 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:05.780984    2570 scope.go:117] "RemoveContainer" containerID="971e8608f5fe57edbebb6fe8d5e17786fdf273c2fed22e3af6cbcfc4cbf11b0d"
Jul 02 13:19:06.786514 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:06.786488    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:19:06.786891 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:06.786594    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-z4gzj" event={"ID":"ed6ac0b5-82f0-4681-ab35-abe5a4a50d80","Type":"ContainerStarted","Data":"44d550008f038d917460122558ab2617abd17b9c5ce753f0586238b5798f6e7b"}
Jul 02 13:19:07.202677 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:07.202640    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:07.263713 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:07.263678    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:07.737475 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:07.737442    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:08.757454 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:08.757425    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-h66tv"
Jul 02 13:19:08.827817 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:08.827791    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:19:08.828134 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:08.828116    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:19:08.833247 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:08.833227    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:19:08.833409 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:08.833380    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:19:08.839860 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:08.839836    2570 kubelet.go:1648] "Image garbage collection succeeded"
Jul 02 13:19:09.341950 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.341902    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:09.377610 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.377576    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mpvj8"]
Jul 02 13:19:09.381083 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.381063    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.383462 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.383441    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 02 13:19:09.383619 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.383595    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 02 13:19:09.383694 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.383596    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vl9hz\""
Jul 02 13:19:09.383694 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.383642    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 02 13:19:09.433621 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.433580    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:09.495536 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.495500    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:09.536554 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.536518    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c605dce2-9e15-479c-a2bf-30cd447ccc0d-metrics-client-ca\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.536554 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.536553    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c605dce2-9e15-479c-a2bf-30cd447ccc0d-node-exporter-wtmp\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.536796 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.536585    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c605dce2-9e15-479c-a2bf-30cd447ccc0d-root\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.536796 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.536634    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c605dce2-9e15-479c-a2bf-30cd447ccc0d-node-exporter-textfile\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.536796 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.536686    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c605dce2-9e15-479c-a2bf-30cd447ccc0d-node-exporter-tls\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.536796 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.536723    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c605dce2-9e15-479c-a2bf-30cd447ccc0d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.537007 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.536810    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c605dce2-9e15-479c-a2bf-30cd447ccc0d-node-exporter-accelerators-collector-config\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.537007 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.536842    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w5cr\" (UniqueName: \"kubernetes.io/projected/c605dce2-9e15-479c-a2bf-30cd447ccc0d-kube-api-access-7w5cr\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.537007 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.536875    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c605dce2-9e15-479c-a2bf-30cd447ccc0d-sys\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.596416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.596309    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:09.637499 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.637456    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c605dce2-9e15-479c-a2bf-30cd447ccc0d-node-exporter-accelerators-collector-config\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.637499 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.637505    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w5cr\" (UniqueName: \"kubernetes.io/projected/c605dce2-9e15-479c-a2bf-30cd447ccc0d-kube-api-access-7w5cr\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.637738 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.637537    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c605dce2-9e15-479c-a2bf-30cd447ccc0d-sys\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.637738 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.637562    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c605dce2-9e15-479c-a2bf-30cd447ccc0d-metrics-client-ca\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.637738 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.637584    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c605dce2-9e15-479c-a2bf-30cd447ccc0d-node-exporter-wtmp\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.637738 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.637629    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c605dce2-9e15-479c-a2bf-30cd447ccc0d-sys\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.637738 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.637647    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c605dce2-9e15-479c-a2bf-30cd447ccc0d-root\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.637738 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.637687    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c605dce2-9e15-479c-a2bf-30cd447ccc0d-root\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.637738 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.637689    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c605dce2-9e15-479c-a2bf-30cd447ccc0d-node-exporter-textfile\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.637738 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.637720    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c605dce2-9e15-479c-a2bf-30cd447ccc0d-node-exporter-wtmp\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.637738 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.637730    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c605dce2-9e15-479c-a2bf-30cd447ccc0d-node-exporter-tls\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.638204 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.637774    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c605dce2-9e15-479c-a2bf-30cd447ccc0d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.638204 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.638005    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c605dce2-9e15-479c-a2bf-30cd447ccc0d-node-exporter-textfile\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.638204 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.638172    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c605dce2-9e15-479c-a2bf-30cd447ccc0d-node-exporter-accelerators-collector-config\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.638332 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.638243    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c605dce2-9e15-479c-a2bf-30cd447ccc0d-metrics-client-ca\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.640180 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.640153    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c605dce2-9e15-479c-a2bf-30cd447ccc0d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.640293 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.640214    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c605dce2-9e15-479c-a2bf-30cd447ccc0d-node-exporter-tls\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.644263 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.644240    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w5cr\" (UniqueName: \"kubernetes.io/projected/c605dce2-9e15-479c-a2bf-30cd447ccc0d-kube-api-access-7w5cr\") pod \"node-exporter-mpvj8\" (UID: \"c605dce2-9e15-479c-a2bf-30cd447ccc0d\") " pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.692153 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.692120    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mpvj8"
Jul 02 13:19:09.696642 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.696509    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:09.703904 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:19:09.703877    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc605dce2_9e15_479c_a2bf_30cd447ccc0d.slice/crio-a43b380842de20c2c859b31450b1e08bdbb227fff15d9e4e17e543687cfb2d16 WatchSource:0}: Error finding container a43b380842de20c2c859b31450b1e08bdbb227fff15d9e4e17e543687cfb2d16: Status 404 returned error can't find the container with id a43b380842de20c2c859b31450b1e08bdbb227fff15d9e4e17e543687cfb2d16
Jul 02 13:19:09.797129 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.797085    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:09.798544 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.798511    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mpvj8" event={"ID":"c605dce2-9e15-479c-a2bf-30cd447ccc0d","Type":"ContainerStarted","Data":"a43b380842de20c2c859b31450b1e08bdbb227fff15d9e4e17e543687cfb2d16"}
Jul 02 13:19:09.936505 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:09.936421    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:10.450351 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.450302    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:10.530890 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.530849    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:10.536204 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.536180    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 02 13:19:10.540533 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.540511    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.542637 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.542442    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 02 13:19:10.542637 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.542475    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 02 13:19:10.542637 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.542501    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 02 13:19:10.542637 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.542515    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 02 13:19:10.542965 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.542773    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 02 13:19:10.542965 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.542773    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 02 13:19:10.542965 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.542904    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-hq4mt\""
Jul 02 13:19:10.543143 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.543095    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 02 13:19:10.543271 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.543255    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 02 13:19:10.543661 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.543476    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 02 13:19:10.553599 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.553576    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 02 13:19:10.593005 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.592965    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:10.646890 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.646584    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.646890 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.646637    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9749c51e-14be-4fa9-8122-00ffe30aa2ca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.646890 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.646670    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg2mk\" (UniqueName: \"kubernetes.io/projected/9749c51e-14be-4fa9-8122-00ffe30aa2ca-kube-api-access-tg2mk\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.646890 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.646696    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.646890 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.646815    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.646890 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.646876    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9749c51e-14be-4fa9-8122-00ffe30aa2ca-config-out\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.647344 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.646946    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-config-volume\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.647344 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.646973    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9749c51e-14be-4fa9-8122-00ffe30aa2ca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.647344 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.647056    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9749c51e-14be-4fa9-8122-00ffe30aa2ca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.647344 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.647128    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.647344 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.647192    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9749c51e-14be-4fa9-8122-00ffe30aa2ca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.647344 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.647222    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.647344 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.647258    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-web-config\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.747831 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.747744    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.747831 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.747793    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9749c51e-14be-4fa9-8122-00ffe30aa2ca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.748054 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.747849    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg2mk\" (UniqueName: \"kubernetes.io/projected/9749c51e-14be-4fa9-8122-00ffe30aa2ca-kube-api-access-tg2mk\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.748054 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.747878    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.748054 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.747923    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.748224 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.748085    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9749c51e-14be-4fa9-8122-00ffe30aa2ca-config-out\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.748224 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.748120    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-config-volume\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.748224 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.748151    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9749c51e-14be-4fa9-8122-00ffe30aa2ca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.748224 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.748209    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9749c51e-14be-4fa9-8122-00ffe30aa2ca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.748375 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.748250    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.748375 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.748281    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9749c51e-14be-4fa9-8122-00ffe30aa2ca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.748375 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.748306    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.748375 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.748336    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-web-config\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.748712 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.748687    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9749c51e-14be-4fa9-8122-00ffe30aa2ca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.749150 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.748845    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9749c51e-14be-4fa9-8122-00ffe30aa2ca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.749150 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.748948    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9749c51e-14be-4fa9-8122-00ffe30aa2ca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.752793 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.752277    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.752793 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.752367    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.752793 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.752492    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-config-volume\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.752793 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.752664    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9749c51e-14be-4fa9-8122-00ffe30aa2ca-config-out\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.753538 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.753482    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9749c51e-14be-4fa9-8122-00ffe30aa2ca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.753899 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.753869    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-web-config\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.754248 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.754227    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.754488 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.754445    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.754598 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.754571    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9749c51e-14be-4fa9-8122-00ffe30aa2ca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.756498 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.756476    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg2mk\" (UniqueName: \"kubernetes.io/projected/9749c51e-14be-4fa9-8122-00ffe30aa2ca-kube-api-access-tg2mk\") pod \"alertmanager-main-0\" (UID: \"9749c51e-14be-4fa9-8122-00ffe30aa2ca\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:10.853424 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:10.853373    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 02 13:19:11.015184 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.015146    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 02 13:19:11.017679 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:19:11.017651    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9749c51e_14be_4fa9_8122_00ffe30aa2ca.slice/crio-5b357fb882b3d4b38f03d895e3ed87babd521900b5e141727ed51a19bcd770d4 WatchSource:0}: Error finding container 5b357fb882b3d4b38f03d895e3ed87babd521900b5e141727ed51a19bcd770d4: Status 404 returned error can't find the container with id 5b357fb882b3d4b38f03d895e3ed87babd521900b5e141727ed51a19bcd770d4
Jul 02 13:19:11.019827 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.019808    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 13:19:11.458080 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.458040    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"]
Jul 02 13:19:11.461737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.461717    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.463701 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.463667    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 02 13:19:11.463701 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.463667    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 02 13:19:11.463957 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.463744    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 02 13:19:11.463957 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.463758    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 02 13:19:11.463957 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.463953    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q4w46\""
Jul 02 13:19:11.464114 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.463955    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 02 13:19:11.465060 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.464534    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-diietp75qos4g\""
Jul 02 13:19:11.472694 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.472668    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"]
Jul 02 13:19:11.555784 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.555746    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtr25\" (UniqueName: \"kubernetes.io/projected/8a02d5b3-ea9e-4f73-b2d7-527250090419-kube-api-access-wtr25\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.555946 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.555818    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a02d5b3-ea9e-4f73-b2d7-527250090419-metrics-client-ca\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.555946 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.555875    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.555946 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.555919    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-grpc-tls\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.555946 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.555945    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.556131 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.555970    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.556131 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.555994    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.556131 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.556014    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-thanos-querier-tls\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.657062 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.657027    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-grpc-tls\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.657251 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.657074    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.657251 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.657091    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.657251 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.657152    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.657251 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.657199    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-thanos-querier-tls\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.657251 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.657244    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtr25\" (UniqueName: \"kubernetes.io/projected/8a02d5b3-ea9e-4f73-b2d7-527250090419-kube-api-access-wtr25\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.657527 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.657306    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a02d5b3-ea9e-4f73-b2d7-527250090419-metrics-client-ca\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.657527 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.657369    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.658166 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.658128    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a02d5b3-ea9e-4f73-b2d7-527250090419-metrics-client-ca\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.659632 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.659609    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.659894 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.659868    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-grpc-tls\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.660036 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.660017    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-thanos-querier-tls\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.660146 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.660122    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.660209 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.660149    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.660209 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.660194    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8a02d5b3-ea9e-4f73-b2d7-527250090419-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.665437 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.665416    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtr25\" (UniqueName: \"kubernetes.io/projected/8a02d5b3-ea9e-4f73-b2d7-527250090419-kube-api-access-wtr25\") pod \"thanos-querier-6597fcffbc-wmh4k\" (UID: \"8a02d5b3-ea9e-4f73-b2d7-527250090419\") " pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.775921 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.775886    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:11.808418 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.808368    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9749c51e-14be-4fa9-8122-00ffe30aa2ca","Type":"ContainerStarted","Data":"5b357fb882b3d4b38f03d895e3ed87babd521900b5e141727ed51a19bcd770d4"}
Jul 02 13:19:11.810164 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.810132    2570 generic.go:356] "Generic (PLEG): container finished" podID="c605dce2-9e15-479c-a2bf-30cd447ccc0d" containerID="7b12c6f595eea436ca359367c5765a958d4c22b15653cc1cd0bcb8963a3b5285" exitCode=0
Jul 02 13:19:11.810284 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.810191    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mpvj8" event={"ID":"c605dce2-9e15-479c-a2bf-30cd447ccc0d","Type":"ContainerDied","Data":"7b12c6f595eea436ca359367c5765a958d4c22b15653cc1cd0bcb8963a3b5285"}
Jul 02 13:19:11.926694 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:11.926663    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"]
Jul 02 13:19:11.932606 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:19:11.932583    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a02d5b3_ea9e_4f73_b2d7_527250090419.slice/crio-9ae0b93c312eb405ba938849947722bc84759e4b84882efaba6b39fc4c23fa99 WatchSource:0}: Error finding container 9ae0b93c312eb405ba938849947722bc84759e4b84882efaba6b39fc4c23fa99: Status 404 returned error can't find the container with id 9ae0b93c312eb405ba938849947722bc84759e4b84882efaba6b39fc4c23fa99
Jul 02 13:19:12.385717 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:12.385656    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:12.487216 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:12.487182    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:12.587527 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:12.587416    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:12.817469 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:12.817422    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k" event={"ID":"8a02d5b3-ea9e-4f73-b2d7-527250090419","Type":"ContainerStarted","Data":"9ae0b93c312eb405ba938849947722bc84759e4b84882efaba6b39fc4c23fa99"}
Jul 02 13:19:12.820007 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:12.819982    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mpvj8" event={"ID":"c605dce2-9e15-479c-a2bf-30cd447ccc0d","Type":"ContainerStarted","Data":"e95fabfe4087f3fceb8d2cb06a0061104efb01e767fc6408650c1d7258302185"}
Jul 02 13:19:12.820095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:12.820037    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mpvj8" event={"ID":"c605dce2-9e15-479c-a2bf-30cd447ccc0d","Type":"ContainerStarted","Data":"93ecb830dc55cd0b9f0465376d988ad8d732017aab1efe2e4046572086edec83"}
Jul 02 13:19:12.836764 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:12.836716    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mpvj8" podStartSLOduration=2.59411144 podStartE2EDuration="3.836695533s" totalImagesPullingTime="1.176287482s" totalInitContainerRuntime="66.296611ms" isStatefulPod=true podCreationTimestamp="2026-07-02 13:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:19:12.834864736 +0000 UTC m=+304.457657642" watchObservedRunningTime="2026-07-02 13:19:12.836695533 +0000 UTC m=+304.459488440"
Jul 02 13:19:13.124047 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:13.123947    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:13.224075 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:13.224043    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:13.824639 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:13.824605    2570 generic.go:356] "Generic (PLEG): container finished" podID="9749c51e-14be-4fa9-8122-00ffe30aa2ca" containerID="38de276fb30636aff1f3204c60c06597c148d1c0d41bc19f058b565b8d37b138" exitCode=0
Jul 02 13:19:13.824810 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:13.824697    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9749c51e-14be-4fa9-8122-00ffe30aa2ca","Type":"ContainerDied","Data":"38de276fb30636aff1f3204c60c06597c148d1c0d41bc19f058b565b8d37b138"}
Jul 02 13:19:14.240527 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:14.240474    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:14.247094 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:14.247069    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-7czq5"]
Jul 02 13:19:14.250397 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:14.250366    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-7czq5"
Jul 02 13:19:14.252166 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:14.252150    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-cjr26\""
Jul 02 13:19:14.252238 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:14.252208    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 02 13:19:14.259166 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:14.259140    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-7czq5"]
Jul 02 13:19:14.335714 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:14.335678    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:14.383667 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:14.383641    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd90139e-accb-4666-ac4d-dce1fcbf1cb4-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-7czq5\" (UID: \"fd90139e-accb-4666-ac4d-dce1fcbf1cb4\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-7czq5"
Jul 02 13:19:14.383819 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:14.383697    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzh84\" (UniqueName: \"kubernetes.io/projected/fd90139e-accb-4666-ac4d-dce1fcbf1cb4-kube-api-access-tzh84\") pod \"monitoring-plugin-cd8656b8-7czq5\" (UID: \"fd90139e-accb-4666-ac4d-dce1fcbf1cb4\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-7czq5"
Jul 02 13:19:14.417154 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:14.417124    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:14.484266 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:14.484225    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd90139e-accb-4666-ac4d-dce1fcbf1cb4-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-7czq5\" (UID: \"fd90139e-accb-4666-ac4d-dce1fcbf1cb4\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-7czq5"
Jul 02 13:19:14.484457 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:14.484319    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzh84\" (UniqueName: \"kubernetes.io/projected/fd90139e-accb-4666-ac4d-dce1fcbf1cb4-kube-api-access-tzh84\") pod \"monitoring-plugin-cd8656b8-7czq5\" (UID: \"fd90139e-accb-4666-ac4d-dce1fcbf1cb4\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-7czq5"
Jul 02 13:19:14.487518 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:14.487488    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd90139e-accb-4666-ac4d-dce1fcbf1cb4-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-7czq5\" (UID: \"fd90139e-accb-4666-ac4d-dce1fcbf1cb4\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-7czq5"
Jul 02 13:19:14.491623 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:14.491557    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzh84\" (UniqueName: \"kubernetes.io/projected/fd90139e-accb-4666-ac4d-dce1fcbf1cb4-kube-api-access-tzh84\") pod \"monitoring-plugin-cd8656b8-7czq5\" (UID: \"fd90139e-accb-4666-ac4d-dce1fcbf1cb4\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-7czq5"
Jul 02 13:19:14.559981 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:14.559945    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-7czq5"
Jul 02 13:19:15.210676 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:15.210648    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-7czq5"]
Jul 02 13:19:15.213756 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:19:15.213723    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd90139e_accb_4666_ac4d_dce1fcbf1cb4.slice/crio-333e042144a0c02f67cb7d3e20da3eea09721468364c86e1f7a8a4d177099da8 WatchSource:0}: Error finding container 333e042144a0c02f67cb7d3e20da3eea09721468364c86e1f7a8a4d177099da8: Status 404 returned error can't find the container with id 333e042144a0c02f67cb7d3e20da3eea09721468364c86e1f7a8a4d177099da8
Jul 02 13:19:15.332131 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:15.332057    2570 patch_prober.go:28] interesting pod/image-registry-f7c8df46f-cdx4k container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 02 13:19:15.332131 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:15.332113    2570 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k" podUID="bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 13:19:15.679285 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:15.679190    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:15.836450 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:15.836418    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-7czq5" event={"ID":"fd90139e-accb-4666-ac4d-dce1fcbf1cb4","Type":"ContainerStarted","Data":"333e042144a0c02f67cb7d3e20da3eea09721468364c86e1f7a8a4d177099da8"}
Jul 02 13:19:15.838974 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:15.838952    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k" event={"ID":"8a02d5b3-ea9e-4f73-b2d7-527250090419","Type":"ContainerStarted","Data":"d9bf809cbfd778ddd6abee7acca6b8d44e0e7758acd9a0d66ec47e1d9bdbff4d"}
Jul 02 13:19:15.839084 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:15.839005    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k" event={"ID":"8a02d5b3-ea9e-4f73-b2d7-527250090419","Type":"ContainerStarted","Data":"47f7e2673a3b0801d1b6a0581a7e0109345d432d22f2ec0c62336ed8b3878f70"}
Jul 02 13:19:15.839084 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:15.839031    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k" event={"ID":"8a02d5b3-ea9e-4f73-b2d7-527250090419","Type":"ContainerStarted","Data":"24f5d21177a59b129af17470596f651e5a483847b7b4e83477a91080e6f75653"}
Jul 02 13:19:16.147839 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:16.147784    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:16.244582 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:16.244535    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:16.746459 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:16.746424    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:19:16.846980 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:16.846889    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9749c51e-14be-4fa9-8122-00ffe30aa2ca","Type":"ContainerStarted","Data":"d6ea7dc04c78093af92ea91891be04bd2d7efba949c60243052574aed7ada655"}
Jul 02 13:19:16.846980 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:16.846977    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9749c51e-14be-4fa9-8122-00ffe30aa2ca","Type":"ContainerStarted","Data":"d1ba51a2faa0067fb939e385fc265b4284581b815b9ffdac116db929cae7d7ce"}
Jul 02 13:19:16.847206 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:16.847020    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9749c51e-14be-4fa9-8122-00ffe30aa2ca","Type":"ContainerStarted","Data":"7344c1efea39e009edddc6c9e2d5fcd82bbf97981fb555efa67ee6a59450e1ef"}
Jul 02 13:19:16.847206 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:16.847051    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9749c51e-14be-4fa9-8122-00ffe30aa2ca","Type":"ContainerStarted","Data":"acb1c812b7aaba77fb153d30eaa693b0fff2be36f12bafb9b488e9df87497e0c"}
Jul 02 13:19:16.847206 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:16.847076    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9749c51e-14be-4fa9-8122-00ffe30aa2ca","Type":"ContainerStarted","Data":"a6fb5fe4d556960a1a80aa05c01c1a971fd84b9c492860f9cd7679da8f13c48f"}
Jul 02 13:19:16.850165 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:16.850135    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k" event={"ID":"8a02d5b3-ea9e-4f73-b2d7-527250090419","Type":"ContainerStarted","Data":"524bed27cac98630ee7847baf640f20bcc7db2ce01bc389c7ac998974481bb74"}
Jul 02 13:19:16.850301 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:16.850206    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k" event={"ID":"8a02d5b3-ea9e-4f73-b2d7-527250090419","Type":"ContainerStarted","Data":"d71f17b91fb9cac16864c50adc16550c50a3fb505a8cf68538bf48080d487e6d"}
Jul 02 13:19:16.850301 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:16.850244    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k" event={"ID":"8a02d5b3-ea9e-4f73-b2d7-527250090419","Type":"ContainerStarted","Data":"c32f1e1e5d0c1a21f207b01a441f5123adad0be94ebff30c5ddc7c52b689cc33"}
Jul 02 13:19:16.850441 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:16.850343    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:16.871101 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:16.871048    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k" podStartSLOduration=1.5575853309999999 podStartE2EDuration="5.871026428s" totalImagesPullingTime="4.313441097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:19:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:19:16.868823118 +0000 UTC m=+308.491616015" watchObservedRunningTime="2026-07-02 13:19:16.871026428 +0000 UTC m=+308.493819338"
Jul 02 13:19:16.949523 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:16.949481    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:16.998976 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:16.998937    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:17.855446 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:17.854636    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f7c8df46f-cdx4k"]
Jul 02 13:19:17.863146 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:17.863111    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9749c51e-14be-4fa9-8122-00ffe30aa2ca","Type":"ContainerStarted","Data":"cf30c52c5e84594cdbbb080bb53ff2d57e30d305471d9204266a6d0f54cafa22"}
Jul 02 13:19:17.865206 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:17.865180    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-7czq5" event={"ID":"fd90139e-accb-4666-ac4d-dce1fcbf1cb4","Type":"ContainerStarted","Data":"3fb9e5a67b2e5a515cbe94676a921e7a5d9ec2d72a7adddc8978245f99d6da81"}
Jul 02 13:19:17.865581 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:17.865554    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-7czq5"
Jul 02 13:19:17.870294 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:17.870274    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-7czq5"
Jul 02 13:19:17.886182 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:17.886155    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:17.891985 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:17.891968    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:19:17.903366 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:17.903324    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.1389369 podStartE2EDuration="7.9033107s" totalImagesPullingTime="4.746049077s" totalInitContainerRuntime="18.324723ms" isStatefulPod=false podCreationTimestamp="2026-07-02 13:19:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:19:17.902992788 +0000 UTC m=+309.525785693" watchObservedRunningTime="2026-07-02 13:19:17.9033107 +0000 UTC m=+309.526103606"
Jul 02 13:19:17.920141 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:17.920081    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-cd8656b8-7czq5" podStartSLOduration=2.003903432 podStartE2EDuration="3.92006141s" totalImagesPullingTime="1.916157978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:19:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:19:17.918263249 +0000 UTC m=+309.541056158" watchObservedRunningTime="2026-07-02 13:19:17.92006141 +0000 UTC m=+309.542854317"
Jul 02 13:19:17.982479 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:17.982435    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:18.084097 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:18.084039    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:18.170080 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:18.169994    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:21.165404 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:21.165340    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:21.243217 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:21.243187    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:22.871635 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:22.871609    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6597fcffbc-wmh4k"
Jul 02 13:19:28.542375 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:28.542318    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:28.601227 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:28.601191    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:28.752897 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:28.752851    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:29.076571 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:29.076531    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:29.963109 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:29.963076    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:30.013771 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:30.013742    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:37.617648 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:37.617594    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:37.809281 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:37.809241    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:39.765749 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:39.765714    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:39.823440 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:39.823408    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:42.876516 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:42.876483    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k" podUID="bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7" containerName="registry" containerID="cri-o://097980bb9773183b525fe9d2498b0397250dde5d9786b7a002863a80c533ba06" gracePeriod=30
Jul 02 13:19:43.124782 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.124759    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:19:43.233700 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.233657    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-image-registry-private-configuration\") pod \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") "
Jul 02 13:19:43.233880 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.233722    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-ca-trust-extracted\") pod \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") "
Jul 02 13:19:43.233880 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.233751    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-installation-pull-secrets\") pod \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") "
Jul 02 13:19:43.233880 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.233777    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-certificates\") pod \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") "
Jul 02 13:19:43.233880 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.233824    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls\") pod \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") "
Jul 02 13:19:43.233880 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.233868    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-trusted-ca\") pod \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") "
Jul 02 13:19:43.234137 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.233899    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prztc\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-kube-api-access-prztc\") pod \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") "
Jul 02 13:19:43.234137 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.233960    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-bound-sa-token\") pod \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\" (UID: \"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7\") "
Jul 02 13:19:43.234578 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.234357    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:19:43.234716 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.234663    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:19:43.236276 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.236246    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:19:43.236469 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.236440    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:19:43.236597 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.236469    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-kube-api-access-prztc" (OuterVolumeSpecName: "kube-api-access-prztc") pod "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7"). InnerVolumeSpecName "kube-api-access-prztc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:19:43.236661 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.236595    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:19:43.236715 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.236671    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:19:43.242189 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.242159    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7" (UID: "bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 13:19:43.335489 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.335451    2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-certificates\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:19:43.335489 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.335482    2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-registry-tls\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:19:43.335489 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.335492    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-trusted-ca\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:19:43.335489 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.335500    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prztc\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-kube-api-access-prztc\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:19:43.335759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.335510    2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-bound-sa-token\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:19:43.335759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.335520    2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-image-registry-private-configuration\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:19:43.335759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.335528    2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-ca-trust-extracted\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:19:43.335759 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.335537    2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7-installation-pull-secrets\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:19:43.972062 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.972028    2570 generic.go:356] "Generic (PLEG): container finished" podID="bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7" containerID="097980bb9773183b525fe9d2498b0397250dde5d9786b7a002863a80c533ba06" exitCode=0
Jul 02 13:19:43.972474 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.972079    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k" event={"ID":"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7","Type":"ContainerDied","Data":"097980bb9773183b525fe9d2498b0397250dde5d9786b7a002863a80c533ba06"}
Jul 02 13:19:43.972474 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.972085    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k"
Jul 02 13:19:43.972474 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.972120    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f7c8df46f-cdx4k" event={"ID":"bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7","Type":"ContainerDied","Data":"a94fa8cc509649d5fee5123f9943de818dfa428f24cd5508626f304b0c7dd845"}
Jul 02 13:19:43.972474 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.972139    2570 scope.go:117] "RemoveContainer" containerID="097980bb9773183b525fe9d2498b0397250dde5d9786b7a002863a80c533ba06"
Jul 02 13:19:43.981025 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.981004    2570 scope.go:117] "RemoveContainer" containerID="097980bb9773183b525fe9d2498b0397250dde5d9786b7a002863a80c533ba06"
Jul 02 13:19:43.981328 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:19:43.981309    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"097980bb9773183b525fe9d2498b0397250dde5d9786b7a002863a80c533ba06\": container with ID starting with 097980bb9773183b525fe9d2498b0397250dde5d9786b7a002863a80c533ba06 not found: ID does not exist" containerID="097980bb9773183b525fe9d2498b0397250dde5d9786b7a002863a80c533ba06"
Jul 02 13:19:43.981469 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.981343    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"097980bb9773183b525fe9d2498b0397250dde5d9786b7a002863a80c533ba06"} err="failed to get container status \"097980bb9773183b525fe9d2498b0397250dde5d9786b7a002863a80c533ba06\": rpc error: code = NotFound desc = could not find container \"097980bb9773183b525fe9d2498b0397250dde5d9786b7a002863a80c533ba06\": container with ID starting with 097980bb9773183b525fe9d2498b0397250dde5d9786b7a002863a80c533ba06 not found: ID does not exist"
Jul 02 13:19:43.991690 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.991661    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f7c8df46f-cdx4k"]
Jul 02 13:19:43.995327 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:43.995307    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-f7c8df46f-cdx4k"]
Jul 02 13:19:44.896975 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:44.896941    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7" path="/var/lib/kubelet/pods/bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7/volumes"
Jul 02 13:19:50.999697 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:50.999660    2570 generic.go:356] "Generic (PLEG): container finished" podID="29eede9f-c968-489b-82c6-12242f0b1801" containerID="50c177681fde737df2b1a534b962d20432b6458c338c9994e96be2274dbfdd0d" exitCode=0
Jul 02 13:19:51.000110 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:50.999730    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9" event={"ID":"29eede9f-c968-489b-82c6-12242f0b1801","Type":"ContainerDied","Data":"50c177681fde737df2b1a534b962d20432b6458c338c9994e96be2274dbfdd0d"}
Jul 02 13:19:51.000110 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:51.000044    2570 scope.go:117] "RemoveContainer" containerID="50c177681fde737df2b1a534b962d20432b6458c338c9994e96be2274dbfdd0d"
Jul 02 13:19:52.005003 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:52.004967    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-2xqx9" event={"ID":"29eede9f-c968-489b-82c6-12242f0b1801","Type":"ContainerStarted","Data":"25c08b406b4711a3ee0c14e9bb18383eb971b85219acf2b193e835b38e32d500"}
Jul 02 13:19:54.775629 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:54.775587    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:19:54.873733 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:19:54.873701    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:01.043678 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:01.043644    2570 generic.go:356] "Generic (PLEG): container finished" podID="273bcb09-ea54-4145-af21-7392e86cdb9b" containerID="c7c4f96d94087ed2133b8f8083f5b1b41c9a97ec028e34818794c75e61b9b657" exitCode=0
Jul 02 13:20:01.044093 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:01.043719    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-847dx" event={"ID":"273bcb09-ea54-4145-af21-7392e86cdb9b","Type":"ContainerDied","Data":"c7c4f96d94087ed2133b8f8083f5b1b41c9a97ec028e34818794c75e61b9b657"}
Jul 02 13:20:01.044093 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:01.044041    2570 scope.go:117] "RemoveContainer" containerID="c7c4f96d94087ed2133b8f8083f5b1b41c9a97ec028e34818794c75e61b9b657"
Jul 02 13:20:02.050060 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:02.050027    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-847dx" event={"ID":"273bcb09-ea54-4145-af21-7392e86cdb9b","Type":"ContainerStarted","Data":"0effae37909eebd872dc7d20b848f3321a8f8c4e342576e4f9f74f80234323c5"}
Jul 02 13:20:02.387486 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:02.387365    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:02.435149 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:02.435112    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:22.884967 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:22.884928    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs\") pod \"network-metrics-daemon-6nhcs\" (UID: \"aa2c8da0-b138-46ef-b9eb-10f6922697be\") " pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:20:22.887212 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:22.887182    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa2c8da0-b138-46ef-b9eb-10f6922697be-metrics-certs\") pod \"network-metrics-daemon-6nhcs\" (UID: \"aa2c8da0-b138-46ef-b9eb-10f6922697be\") " pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:20:22.995441 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:22.995406    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k9zxt\""
Jul 02 13:20:23.003824 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:23.003798    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6nhcs"
Jul 02 13:20:23.129240 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:23.129215    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6nhcs"]
Jul 02 13:20:23.131634 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:20:23.131592    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa2c8da0_b138_46ef_b9eb_10f6922697be.slice/crio-6564866eb6098d521c64ddfd239fa04fe5600bb21d5cc83ea2438c9b7474bace WatchSource:0}: Error finding container 6564866eb6098d521c64ddfd239fa04fe5600bb21d5cc83ea2438c9b7474bace: Status 404 returned error can't find the container with id 6564866eb6098d521c64ddfd239fa04fe5600bb21d5cc83ea2438c9b7474bace
Jul 02 13:20:24.140740 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:24.140680    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6nhcs" event={"ID":"aa2c8da0-b138-46ef-b9eb-10f6922697be","Type":"ContainerStarted","Data":"6564866eb6098d521c64ddfd239fa04fe5600bb21d5cc83ea2438c9b7474bace"}
Jul 02 13:20:25.146754 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:25.146715    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6nhcs" event={"ID":"aa2c8da0-b138-46ef-b9eb-10f6922697be","Type":"ContainerStarted","Data":"dd8dac400f21b273f5d940f251893010350317ee86e9214726f99cd4e5157f59"}
Jul 02 13:20:25.147227 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:25.146769    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6nhcs" event={"ID":"aa2c8da0-b138-46ef-b9eb-10f6922697be","Type":"ContainerStarted","Data":"12d8be8c440e087a53c8823a9beafee08023f1e5770eb8fef40e727671b783c8"}
Jul 02 13:20:25.161737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:25.161683    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6nhcs" podStartSLOduration=375.027450548 podStartE2EDuration="6m16.161668216s" totalImagesPullingTime="1.134217668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:14:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:20:25.160147761 +0000 UTC m=+376.782940667" watchObservedRunningTime="2026-07-02 13:20:25.161668216 +0000 UTC m=+376.784461122"
Jul 02 13:20:25.241483 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:25.241442    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:25.294473 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:25.294445    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:28.661177 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:28.661132    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:29.168466 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:29.168428    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:33.752249 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.752202    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-675f768f9b-89m2k"]
Jul 02 13:20:33.752870 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.752726    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7" containerName="registry"
Jul 02 13:20:33.752870 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.752744    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7" containerName="registry"
Jul 02 13:20:33.752870 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.752831    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcb9e608-ff8c-4afc-8e6d-36d1b416b4c7" containerName="registry"
Jul 02 13:20:33.756447 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.756423    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.759046 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.759021    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 02 13:20:33.759192 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.759106    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-s72nf\""
Jul 02 13:20:33.759288 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.759269    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 02 13:20:33.759546 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.759528    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 02 13:20:33.759662 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.759606    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 02 13:20:33.759738 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.759685    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 02 13:20:33.766206 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.765103    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 02 13:20:33.771160 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.771132    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-675f768f9b-89m2k"]
Jul 02 13:20:33.881929 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.881894    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c07a06ee-5118-4201-8a60-b2d7ec5270c0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.882116 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.881952    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c07a06ee-5118-4201-8a60-b2d7ec5270c0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.882116 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.882017    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c07a06ee-5118-4201-8a60-b2d7ec5270c0-federate-client-tls\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.882116 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.882038    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c07a06ee-5118-4201-8a60-b2d7ec5270c0-secret-telemeter-client\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.882116 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.882079    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c07a06ee-5118-4201-8a60-b2d7ec5270c0-telemeter-client-tls\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.882248 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.882114    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t572q\" (UniqueName: \"kubernetes.io/projected/c07a06ee-5118-4201-8a60-b2d7ec5270c0-kube-api-access-t572q\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.882248 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.882144    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c07a06ee-5118-4201-8a60-b2d7ec5270c0-metrics-client-ca\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.882248 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.882160    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c07a06ee-5118-4201-8a60-b2d7ec5270c0-serving-certs-ca-bundle\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.983439 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.983361    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c07a06ee-5118-4201-8a60-b2d7ec5270c0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.983637 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.983470    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c07a06ee-5118-4201-8a60-b2d7ec5270c0-federate-client-tls\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.983637 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.983499    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c07a06ee-5118-4201-8a60-b2d7ec5270c0-secret-telemeter-client\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.983637 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.983535    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c07a06ee-5118-4201-8a60-b2d7ec5270c0-telemeter-client-tls\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.983637 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.983584    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t572q\" (UniqueName: \"kubernetes.io/projected/c07a06ee-5118-4201-8a60-b2d7ec5270c0-kube-api-access-t572q\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.983637 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.983612    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c07a06ee-5118-4201-8a60-b2d7ec5270c0-metrics-client-ca\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.983637 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.983635    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c07a06ee-5118-4201-8a60-b2d7ec5270c0-serving-certs-ca-bundle\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.983977 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.983676    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c07a06ee-5118-4201-8a60-b2d7ec5270c0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.984447 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.984420    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c07a06ee-5118-4201-8a60-b2d7ec5270c0-metrics-client-ca\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.984551 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.984500    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c07a06ee-5118-4201-8a60-b2d7ec5270c0-serving-certs-ca-bundle\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.984675 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.984656    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c07a06ee-5118-4201-8a60-b2d7ec5270c0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.986142 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.986121    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c07a06ee-5118-4201-8a60-b2d7ec5270c0-secret-telemeter-client\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.986142 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.986130    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c07a06ee-5118-4201-8a60-b2d7ec5270c0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.986370 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.986353    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c07a06ee-5118-4201-8a60-b2d7ec5270c0-telemeter-client-tls\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.986525 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.986504    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c07a06ee-5118-4201-8a60-b2d7ec5270c0-federate-client-tls\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:33.990626 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:33.990603    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t572q\" (UniqueName: \"kubernetes.io/projected/c07a06ee-5118-4201-8a60-b2d7ec5270c0-kube-api-access-t572q\") pod \"telemeter-client-675f768f9b-89m2k\" (UID: \"c07a06ee-5118-4201-8a60-b2d7ec5270c0\") " pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:34.070761 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:34.070673    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k"
Jul 02 13:20:34.205792 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:34.205741    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-675f768f9b-89m2k"]
Jul 02 13:20:34.208881 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:20:34.208851    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc07a06ee_5118_4201_8a60_b2d7ec5270c0.slice/crio-82960c435ae6a46f01b79cbab2c25b1483f179ab21928b1038866ca2fe58294d WatchSource:0}: Error finding container 82960c435ae6a46f01b79cbab2c25b1483f179ab21928b1038866ca2fe58294d: Status 404 returned error can't find the container with id 82960c435ae6a46f01b79cbab2c25b1483f179ab21928b1038866ca2fe58294d
Jul 02 13:20:35.188506 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:35.188469    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k" event={"ID":"c07a06ee-5118-4201-8a60-b2d7ec5270c0","Type":"ContainerStarted","Data":"82960c435ae6a46f01b79cbab2c25b1483f179ab21928b1038866ca2fe58294d"}
Jul 02 13:20:36.261436 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:36.261404    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:37.199068 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:37.199036    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k" event={"ID":"c07a06ee-5118-4201-8a60-b2d7ec5270c0","Type":"ContainerStarted","Data":"a484ecd7230ac8f25351a2dd4a807b33e62822f05962ac2649c774b8f2c746de"}
Jul 02 13:20:37.199239 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:37.199090    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k" event={"ID":"c07a06ee-5118-4201-8a60-b2d7ec5270c0","Type":"ContainerStarted","Data":"485723cc89d9c1a0b09454f461cef98688d8aaefba142d06873e39663272e9f0"}
Jul 02 13:20:37.199239 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:37.199114    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k" event={"ID":"c07a06ee-5118-4201-8a60-b2d7ec5270c0","Type":"ContainerStarted","Data":"65c391116ba6e0200259a3ac39b78e16a3af46cd8912c367c3c0390220489560"}
Jul 02 13:20:37.216925 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:37.216875    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-675f768f9b-89m2k" podStartSLOduration=2.133979806 podStartE2EDuration="4.216859363s" totalImagesPullingTime="2.082879557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:20:37.216213583 +0000 UTC m=+388.839006490" watchObservedRunningTime="2026-07-02 13:20:37.216859363 +0000 UTC m=+388.839652304"
Jul 02 13:20:38.006829 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:38.006795    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:38.172485 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:38.172445    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:38.655090 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:38.655054    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:38.707792 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:38.707758    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:39.258721 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:39.258692    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:39.319003 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:39.318973    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:43.892753 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:43.892713    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:43.993348 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:43.993305    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:53.884926 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:53.884894    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:55.869798 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:55.869766    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:55.937020 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:55.936976    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:58.698047 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:58.698013    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:58.793321 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:58.793286    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:20:58.841430 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:20:58.841372    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:21:08.560866 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:21:08.560824    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:21:09.783454 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:21:09.783418    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:21:09.881788 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:21:09.881754    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:21:31.612803 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:21:31.612717    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:22:07.881754 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:22:07.881723    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:22:07.982118 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:22:07.982085    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:22:19.184827 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:22:19.184791    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:04.312156 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:04.312069    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:04.412650 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:04.412608    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:04.640854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:04.640768    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:04.697557 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:04.697523    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:04.758129 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:04.758099    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:26.350649 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:26.350610    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:26.474251 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:26.474213    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:33.144600 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.144561    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:33.413517 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.413427    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:33.588737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.588700    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:33.628047 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.628015    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"]
Jul 02 13:23:33.631173 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.631156    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"
Jul 02 13:23:33.633110 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.633090    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 02 13:23:33.633223 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.633122    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 02 13:23:33.633223 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.633096    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 02 13:23:33.633367 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.633348    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 02 13:23:33.633504 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.633449    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-xg4zl\""
Jul 02 13:23:33.633504 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.633483    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 02 13:23:33.640705 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.640684    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"]
Jul 02 13:23:33.749942 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.749886    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:33.769052 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.769013    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/27f24566-db7b-4441-9c99-f5acaf8f6ddc-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-r2vtb\" (UID: \"27f24566-db7b-4441-9c99-f5acaf8f6ddc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"
Jul 02 13:23:33.769224 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.769072    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/27f24566-db7b-4441-9c99-f5acaf8f6ddc-certificates\") pod \"keda-metrics-apiserver-559f75f947-r2vtb\" (UID: \"27f24566-db7b-4441-9c99-f5acaf8f6ddc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"
Jul 02 13:23:33.769224 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.769155    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzd5f\" (UniqueName: \"kubernetes.io/projected/27f24566-db7b-4441-9c99-f5acaf8f6ddc-kube-api-access-tzd5f\") pod \"keda-metrics-apiserver-559f75f947-r2vtb\" (UID: \"27f24566-db7b-4441-9c99-f5acaf8f6ddc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"
Jul 02 13:23:33.870237 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.870195    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/27f24566-db7b-4441-9c99-f5acaf8f6ddc-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-r2vtb\" (UID: \"27f24566-db7b-4441-9c99-f5acaf8f6ddc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"
Jul 02 13:23:33.870427 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.870247    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/27f24566-db7b-4441-9c99-f5acaf8f6ddc-certificates\") pod \"keda-metrics-apiserver-559f75f947-r2vtb\" (UID: \"27f24566-db7b-4441-9c99-f5acaf8f6ddc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"
Jul 02 13:23:33.870427 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.870325    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzd5f\" (UniqueName: \"kubernetes.io/projected/27f24566-db7b-4441-9c99-f5acaf8f6ddc-kube-api-access-tzd5f\") pod \"keda-metrics-apiserver-559f75f947-r2vtb\" (UID: \"27f24566-db7b-4441-9c99-f5acaf8f6ddc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"
Jul 02 13:23:33.870508 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:23:33.870454    2570 secret.go:281] references non-existent secret key: tls.crt
Jul 02 13:23:33.870508 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:23:33.870475    2570 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jul 02 13:23:33.870508 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:23:33.870491    2570 projected.go:266] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found
Jul 02 13:23:33.870607 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:23:33.870509    2570 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb: [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found]
Jul 02 13:23:33.870607 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:23:33.870577    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27f24566-db7b-4441-9c99-f5acaf8f6ddc-certificates podName:27f24566-db7b-4441-9c99-f5acaf8f6ddc nodeName:}" failed. No retries permitted until 2026-07-02 13:23:34.37055685 +0000 UTC m=+565.993349746 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/27f24566-db7b-4441-9c99-f5acaf8f6ddc-certificates") pod "keda-metrics-apiserver-559f75f947-r2vtb" (UID: "27f24566-db7b-4441-9c99-f5acaf8f6ddc") : [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found]
Jul 02 13:23:33.870703 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.870681    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/27f24566-db7b-4441-9c99-f5acaf8f6ddc-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-r2vtb\" (UID: \"27f24566-db7b-4441-9c99-f5acaf8f6ddc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"
Jul 02 13:23:33.881739 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:33.881716    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzd5f\" (UniqueName: \"kubernetes.io/projected/27f24566-db7b-4441-9c99-f5acaf8f6ddc-kube-api-access-tzd5f\") pod \"keda-metrics-apiserver-559f75f947-r2vtb\" (UID: \"27f24566-db7b-4441-9c99-f5acaf8f6ddc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"
Jul 02 13:23:34.010064 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.009975    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:34.099583 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.099550    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:34.237023 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.236984    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-6dfhk"]
Jul 02 13:23:34.240380 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.240363    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-6dfhk"
Jul 02 13:23:34.242146 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.242120    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 02 13:23:34.251374 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.251347    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-6dfhk"]
Jul 02 13:23:34.297653 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.297570    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:34.375656 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.375616    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/39efbec0-d42b-4b2a-846c-69a17618cd37-certificates\") pod \"keda-admission-5fbd5c4644-6dfhk\" (UID: \"39efbec0-d42b-4b2a-846c-69a17618cd37\") " pod="openshift-keda/keda-admission-5fbd5c4644-6dfhk"
Jul 02 13:23:34.375656 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.375656    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8nkk\" (UniqueName: \"kubernetes.io/projected/39efbec0-d42b-4b2a-846c-69a17618cd37-kube-api-access-l8nkk\") pod \"keda-admission-5fbd5c4644-6dfhk\" (UID: \"39efbec0-d42b-4b2a-846c-69a17618cd37\") " pod="openshift-keda/keda-admission-5fbd5c4644-6dfhk"
Jul 02 13:23:34.375886 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.375750    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/27f24566-db7b-4441-9c99-f5acaf8f6ddc-certificates\") pod \"keda-metrics-apiserver-559f75f947-r2vtb\" (UID: \"27f24566-db7b-4441-9c99-f5acaf8f6ddc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"
Jul 02 13:23:34.375886 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:23:34.375860    2570 secret.go:281] references non-existent secret key: tls.crt
Jul 02 13:23:34.375886 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:23:34.375873    2570 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jul 02 13:23:34.375978 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:23:34.375889    2570 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb: references non-existent secret key: tls.crt
Jul 02 13:23:34.375978 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:23:34.375958    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27f24566-db7b-4441-9c99-f5acaf8f6ddc-certificates podName:27f24566-db7b-4441-9c99-f5acaf8f6ddc nodeName:}" failed. No retries permitted until 2026-07-02 13:23:35.375944243 +0000 UTC m=+566.998737127 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/27f24566-db7b-4441-9c99-f5acaf8f6ddc-certificates") pod "keda-metrics-apiserver-559f75f947-r2vtb" (UID: "27f24566-db7b-4441-9c99-f5acaf8f6ddc") : references non-existent secret key: tls.crt
Jul 02 13:23:34.460200 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.460170    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:34.476310 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.476281    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/39efbec0-d42b-4b2a-846c-69a17618cd37-certificates\") pod \"keda-admission-5fbd5c4644-6dfhk\" (UID: \"39efbec0-d42b-4b2a-846c-69a17618cd37\") " pod="openshift-keda/keda-admission-5fbd5c4644-6dfhk"
Jul 02 13:23:34.476443 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.476316    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8nkk\" (UniqueName: \"kubernetes.io/projected/39efbec0-d42b-4b2a-846c-69a17618cd37-kube-api-access-l8nkk\") pod \"keda-admission-5fbd5c4644-6dfhk\" (UID: \"39efbec0-d42b-4b2a-846c-69a17618cd37\") " pod="openshift-keda/keda-admission-5fbd5c4644-6dfhk"
Jul 02 13:23:34.478899 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.478879    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/39efbec0-d42b-4b2a-846c-69a17618cd37-certificates\") pod \"keda-admission-5fbd5c4644-6dfhk\" (UID: \"39efbec0-d42b-4b2a-846c-69a17618cd37\") " pod="openshift-keda/keda-admission-5fbd5c4644-6dfhk"
Jul 02 13:23:34.483512 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.483492    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8nkk\" (UniqueName: \"kubernetes.io/projected/39efbec0-d42b-4b2a-846c-69a17618cd37-kube-api-access-l8nkk\") pod \"keda-admission-5fbd5c4644-6dfhk\" (UID: \"39efbec0-d42b-4b2a-846c-69a17618cd37\") " pod="openshift-keda/keda-admission-5fbd5c4644-6dfhk"
Jul 02 13:23:34.552494 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.552380    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-6dfhk"
Jul 02 13:23:34.561670 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.561641    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:34.685126 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.685078    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-6dfhk"]
Jul 02 13:23:34.687396 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:23:34.687354    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39efbec0_d42b_4b2a_846c_69a17618cd37.slice/crio-2290a29d32ae7f798007f6690a9509bb1b186b4577dd98004843e5b1d7ac43e5 WatchSource:0}: Error finding container 2290a29d32ae7f798007f6690a9509bb1b186b4577dd98004843e5b1d7ac43e5: Status 404 returned error can't find the container with id 2290a29d32ae7f798007f6690a9509bb1b186b4577dd98004843e5b1d7ac43e5
Jul 02 13:23:34.916118 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:34.916024    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-6dfhk" event={"ID":"39efbec0-d42b-4b2a-846c-69a17618cd37","Type":"ContainerStarted","Data":"2290a29d32ae7f798007f6690a9509bb1b186b4577dd98004843e5b1d7ac43e5"}
Jul 02 13:23:35.343845 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:35.343774    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:35.386339 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:35.386113    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/27f24566-db7b-4441-9c99-f5acaf8f6ddc-certificates\") pod \"keda-metrics-apiserver-559f75f947-r2vtb\" (UID: \"27f24566-db7b-4441-9c99-f5acaf8f6ddc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"
Jul 02 13:23:35.389485 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:35.389455    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/27f24566-db7b-4441-9c99-f5acaf8f6ddc-certificates\") pod \"keda-metrics-apiserver-559f75f947-r2vtb\" (UID: \"27f24566-db7b-4441-9c99-f5acaf8f6ddc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"
Jul 02 13:23:35.443338 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:35.443304    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"
Jul 02 13:23:35.455939 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:35.455902    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:35.620257 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:35.618268    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"]
Jul 02 13:23:35.923317 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:35.923228    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb" event={"ID":"27f24566-db7b-4441-9c99-f5acaf8f6ddc","Type":"ContainerStarted","Data":"2fcffb0357614418436dd45da33a758d170d83a79ec9495daecb61babded79a2"}
Jul 02 13:23:36.825466 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:36.825426    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:37.935324 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:37.935281    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-6dfhk" event={"ID":"39efbec0-d42b-4b2a-846c-69a17618cd37","Type":"ContainerStarted","Data":"6b5c572274581941e9ece9efacff67546ef7bdfaf37ee477424e60a15ea1348b"}
Jul 02 13:23:37.935775 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:37.935449    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-6dfhk"
Jul 02 13:23:37.951214 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:37.951150    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-6dfhk" podStartSLOduration=1.553698998 podStartE2EDuration="3.95112866s" totalImagesPullingTime="2.397429662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:23:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:23:37.949113576 +0000 UTC m=+569.571906483" watchObservedRunningTime="2026-07-02 13:23:37.95112866 +0000 UTC m=+569.573921567"
Jul 02 13:23:39.946096 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:39.946061    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb" event={"ID":"27f24566-db7b-4441-9c99-f5acaf8f6ddc","Type":"ContainerStarted","Data":"7e7b887b3f94243862f8a37922275f7dcf92b3e45a6e395ab29824b82ed45b14"}
Jul 02 13:23:39.946493 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:39.946132    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"
Jul 02 13:23:39.961057 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:39.961006    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb" podStartSLOduration=2.975115706 podStartE2EDuration="6.9609888s" totalImagesPullingTime="3.985873094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:23:39.95914865 +0000 UTC m=+571.581941578" watchObservedRunningTime="2026-07-02 13:23:39.9609888 +0000 UTC m=+571.583781705"
Jul 02 13:23:47.588026 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:47.587990    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:50.954784 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:50.954757    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-r2vtb"
Jul 02 13:23:58.626649 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:58.626608    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:23:58.942354 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:23:58.942269    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-6dfhk"
Jul 02 13:24:08.858496 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:08.858464    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:24:08.859051 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:08.858530    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:24:08.864591 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:08.864572    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:24:08.864720 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:08.864657    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:24:09.360693 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:09.360572    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:24:30.730220 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:30.730182    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:24:30.831978 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:30.831942    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:24:32.675587 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:32.675557    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:24:32.722229 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:32.722205    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:24:44.183814 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.183771    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:24:44.267023 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.266986    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:24:44.288792 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.288757    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-nnnjb"]
Jul 02 13:24:44.292126 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.292110    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-nnnjb"
Jul 02 13:24:44.294006 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.293982    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-pzc2x\""
Jul 02 13:24:44.294006 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.294002    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 02 13:24:44.294406 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.294374    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 02 13:24:44.294456 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.294427    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 02 13:24:44.300320 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.300099    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-nnnjb"]
Jul 02 13:24:44.337160 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.337106    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:24:44.437831 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.437732    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m2b6\" (UniqueName: \"kubernetes.io/projected/af9df460-cc0b-4046-b579-e1332211dd14-kube-api-access-5m2b6\") pod \"seaweedfs-64568bcd49-nnnjb\" (UID: \"af9df460-cc0b-4046-b579-e1332211dd14\") " pod="kserve/seaweedfs-64568bcd49-nnnjb"
Jul 02 13:24:44.437831 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.437780    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/af9df460-cc0b-4046-b579-e1332211dd14-data\") pod \"seaweedfs-64568bcd49-nnnjb\" (UID: \"af9df460-cc0b-4046-b579-e1332211dd14\") " pod="kserve/seaweedfs-64568bcd49-nnnjb"
Jul 02 13:24:44.438365 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.438329    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:24:44.539186 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.539143    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m2b6\" (UniqueName: \"kubernetes.io/projected/af9df460-cc0b-4046-b579-e1332211dd14-kube-api-access-5m2b6\") pod \"seaweedfs-64568bcd49-nnnjb\" (UID: \"af9df460-cc0b-4046-b579-e1332211dd14\") " pod="kserve/seaweedfs-64568bcd49-nnnjb"
Jul 02 13:24:44.539415 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.539218    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/af9df460-cc0b-4046-b579-e1332211dd14-data\") pod \"seaweedfs-64568bcd49-nnnjb\" (UID: \"af9df460-cc0b-4046-b579-e1332211dd14\") " pod="kserve/seaweedfs-64568bcd49-nnnjb"
Jul 02 13:24:44.539668 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.539648    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/af9df460-cc0b-4046-b579-e1332211dd14-data\") pod \"seaweedfs-64568bcd49-nnnjb\" (UID: \"af9df460-cc0b-4046-b579-e1332211dd14\") " pod="kserve/seaweedfs-64568bcd49-nnnjb"
Jul 02 13:24:44.545870 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.545845    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m2b6\" (UniqueName: \"kubernetes.io/projected/af9df460-cc0b-4046-b579-e1332211dd14-kube-api-access-5m2b6\") pod \"seaweedfs-64568bcd49-nnnjb\" (UID: \"af9df460-cc0b-4046-b579-e1332211dd14\") " pod="kserve/seaweedfs-64568bcd49-nnnjb"
Jul 02 13:24:44.603292 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.603253    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-nnnjb"
Jul 02 13:24:44.750405 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.750307    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-nnnjb"]
Jul 02 13:24:44.752677 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:24:44.752645    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf9df460_cc0b_4046_b579_e1332211dd14.slice/crio-c3ae1508a6ee11c8d36083c0df1c06a3b8ee9a4f42edb013b5bf9bb235b1cc74 WatchSource:0}: Error finding container c3ae1508a6ee11c8d36083c0df1c06a3b8ee9a4f42edb013b5bf9bb235b1cc74: Status 404 returned error can't find the container with id c3ae1508a6ee11c8d36083c0df1c06a3b8ee9a4f42edb013b5bf9bb235b1cc74
Jul 02 13:24:44.754025 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.754004    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 13:24:44.777394 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.777356    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:24:44.965541 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:44.965498    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:24:45.066960 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:45.066877    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:24:45.123320 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:45.123288    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:24:45.217465 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:45.217429    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-nnnjb" event={"ID":"af9df460-cc0b-4046-b579-e1332211dd14","Type":"ContainerStarted","Data":"c3ae1508a6ee11c8d36083c0df1c06a3b8ee9a4f42edb013b5bf9bb235b1cc74"}
Jul 02 13:24:48.232709 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:48.232678    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-nnnjb" event={"ID":"af9df460-cc0b-4046-b579-e1332211dd14","Type":"ContainerStarted","Data":"9f2764a7261cfb2797cc08a9e44ab1deaddf5d640731b72afbc1f08d279ff28c"}
Jul 02 13:24:48.233072 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:48.232743    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-nnnjb"
Jul 02 13:24:48.245827 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:48.245783    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-nnnjb" podStartSLOduration=1.649305732 podStartE2EDuration="4.245767821s" totalImagesPullingTime="2.596462089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:24:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:24:48.24514432 +0000 UTC m=+639.867937227" watchObservedRunningTime="2026-07-02 13:24:48.245767821 +0000 UTC m=+639.868560727"
Jul 02 13:24:50.498163 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:50.498130    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:24:50.556409 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:50.556351    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:24:54.241566 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:24:54.241536    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-nnnjb"
Jul 02 13:25:47.286607 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:47.286561    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:25:47.335643 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:47.335610    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:25:53.488882 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.488846    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:25:53.533957 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.533926    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-fd65d7d6-gk29w"]
Jul 02 13:25:53.537328 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.537309    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-gk29w"
Jul 02 13:25:53.539289 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.539244    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Jul 02 13:25:53.539557 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.539539    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-9nssx\""
Jul 02 13:25:53.547366 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.547343    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-gk29w"]
Jul 02 13:25:53.550736 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.550714    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-ddtp6"]
Jul 02 13:25:53.554219 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.554199    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-ddtp6"
Jul 02 13:25:53.556181 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.556159    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Jul 02 13:25:53.556181 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.556176    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-22p6c\""
Jul 02 13:25:53.562884 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.562856    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-ddtp6"]
Jul 02 13:25:53.569715 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.569687    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:25:53.617727 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.617691    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94c22db7-55c6-4708-aba0-c2c1c9cc9576-cert\") pod \"odh-model-controller-84b54b7f97-ddtp6\" (UID: \"94c22db7-55c6-4708-aba0-c2c1c9cc9576\") " pod="kserve/odh-model-controller-84b54b7f97-ddtp6"
Jul 02 13:25:53.617900 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.617733    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c99e727e-7b07-4e77-8452-ce73f33cd3f6-tls-certs\") pod \"model-serving-api-fd65d7d6-gk29w\" (UID: \"c99e727e-7b07-4e77-8452-ce73f33cd3f6\") " pod="kserve/model-serving-api-fd65d7d6-gk29w"
Jul 02 13:25:53.617900 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.617822    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jptpt\" (UniqueName: \"kubernetes.io/projected/94c22db7-55c6-4708-aba0-c2c1c9cc9576-kube-api-access-jptpt\") pod \"odh-model-controller-84b54b7f97-ddtp6\" (UID: \"94c22db7-55c6-4708-aba0-c2c1c9cc9576\") " pod="kserve/odh-model-controller-84b54b7f97-ddtp6"
Jul 02 13:25:53.617900 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.617854    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w7k9\" (UniqueName: \"kubernetes.io/projected/c99e727e-7b07-4e77-8452-ce73f33cd3f6-kube-api-access-6w7k9\") pod \"model-serving-api-fd65d7d6-gk29w\" (UID: \"c99e727e-7b07-4e77-8452-ce73f33cd3f6\") " pod="kserve/model-serving-api-fd65d7d6-gk29w"
Jul 02 13:25:53.632885 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.632859    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:25:53.718321 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.718277    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jptpt\" (UniqueName: \"kubernetes.io/projected/94c22db7-55c6-4708-aba0-c2c1c9cc9576-kube-api-access-jptpt\") pod \"odh-model-controller-84b54b7f97-ddtp6\" (UID: \"94c22db7-55c6-4708-aba0-c2c1c9cc9576\") " pod="kserve/odh-model-controller-84b54b7f97-ddtp6"
Jul 02 13:25:53.718533 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.718325    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w7k9\" (UniqueName: \"kubernetes.io/projected/c99e727e-7b07-4e77-8452-ce73f33cd3f6-kube-api-access-6w7k9\") pod \"model-serving-api-fd65d7d6-gk29w\" (UID: \"c99e727e-7b07-4e77-8452-ce73f33cd3f6\") " pod="kserve/model-serving-api-fd65d7d6-gk29w"
Jul 02 13:25:53.718533 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.718463    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94c22db7-55c6-4708-aba0-c2c1c9cc9576-cert\") pod \"odh-model-controller-84b54b7f97-ddtp6\" (UID: \"94c22db7-55c6-4708-aba0-c2c1c9cc9576\") " pod="kserve/odh-model-controller-84b54b7f97-ddtp6"
Jul 02 13:25:53.718533 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.718502    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c99e727e-7b07-4e77-8452-ce73f33cd3f6-tls-certs\") pod \"model-serving-api-fd65d7d6-gk29w\" (UID: \"c99e727e-7b07-4e77-8452-ce73f33cd3f6\") " pod="kserve/model-serving-api-fd65d7d6-gk29w"
Jul 02 13:25:53.718670 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:25:53.718630    2570 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 02 13:25:53.718711 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:25:53.718694    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c99e727e-7b07-4e77-8452-ce73f33cd3f6-tls-certs podName:c99e727e-7b07-4e77-8452-ce73f33cd3f6 nodeName:}" failed. No retries permitted until 2026-07-02 13:25:54.218672438 +0000 UTC m=+705.841465329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/c99e727e-7b07-4e77-8452-ce73f33cd3f6-tls-certs") pod "model-serving-api-fd65d7d6-gk29w" (UID: "c99e727e-7b07-4e77-8452-ce73f33cd3f6") : secret "model-serving-api-tls" not found
Jul 02 13:25:53.721159 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.721134    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94c22db7-55c6-4708-aba0-c2c1c9cc9576-cert\") pod \"odh-model-controller-84b54b7f97-ddtp6\" (UID: \"94c22db7-55c6-4708-aba0-c2c1c9cc9576\") " pod="kserve/odh-model-controller-84b54b7f97-ddtp6"
Jul 02 13:25:53.726330 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.726304    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jptpt\" (UniqueName: \"kubernetes.io/projected/94c22db7-55c6-4708-aba0-c2c1c9cc9576-kube-api-access-jptpt\") pod \"odh-model-controller-84b54b7f97-ddtp6\" (UID: \"94c22db7-55c6-4708-aba0-c2c1c9cc9576\") " pod="kserve/odh-model-controller-84b54b7f97-ddtp6"
Jul 02 13:25:53.726446 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.726332    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w7k9\" (UniqueName: \"kubernetes.io/projected/c99e727e-7b07-4e77-8452-ce73f33cd3f6-kube-api-access-6w7k9\") pod \"model-serving-api-fd65d7d6-gk29w\" (UID: \"c99e727e-7b07-4e77-8452-ce73f33cd3f6\") " pod="kserve/model-serving-api-fd65d7d6-gk29w"
Jul 02 13:25:53.791120 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.791038    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:25:53.866807 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.866773    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-ddtp6"
Jul 02 13:25:53.998337 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:53.998305    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-ddtp6"]
Jul 02 13:25:54.001935 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:25:54.001892    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94c22db7_55c6_4708_aba0_c2c1c9cc9576.slice/crio-7f039d1b163a658fb752d3a7a1c3eadf8b3ae84bf68c3d0d052cc341250ee0c8 WatchSource:0}: Error finding container 7f039d1b163a658fb752d3a7a1c3eadf8b3ae84bf68c3d0d052cc341250ee0c8: Status 404 returned error can't find the container with id 7f039d1b163a658fb752d3a7a1c3eadf8b3ae84bf68c3d0d052cc341250ee0c8
Jul 02 13:25:54.134407 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:54.134281    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:25:54.223133 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:54.223097    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c99e727e-7b07-4e77-8452-ce73f33cd3f6-tls-certs\") pod \"model-serving-api-fd65d7d6-gk29w\" (UID: \"c99e727e-7b07-4e77-8452-ce73f33cd3f6\") " pod="kserve/model-serving-api-fd65d7d6-gk29w"
Jul 02 13:25:54.225510 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:54.225485    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c99e727e-7b07-4e77-8452-ce73f33cd3f6-tls-certs\") pod \"model-serving-api-fd65d7d6-gk29w\" (UID: \"c99e727e-7b07-4e77-8452-ce73f33cd3f6\") " pod="kserve/model-serving-api-fd65d7d6-gk29w"
Jul 02 13:25:54.235564 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:54.235521    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:25:54.448971 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:54.448939    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-gk29w"
Jul 02 13:25:54.519296 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:54.519215    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-ddtp6" event={"ID":"94c22db7-55c6-4708-aba0-c2c1c9cc9576","Type":"ContainerStarted","Data":"7f039d1b163a658fb752d3a7a1c3eadf8b3ae84bf68c3d0d052cc341250ee0c8"}
Jul 02 13:25:54.614705 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:54.614673    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-gk29w"]
Jul 02 13:25:54.618712 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:25:54.618677    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc99e727e_7b07_4e77_8452_ce73f33cd3f6.slice/crio-ffa27ad97e1afa5a0d2f9885cccf55399141b9dca15a516e0b7cc3248f5c2cd7 WatchSource:0}: Error finding container ffa27ad97e1afa5a0d2f9885cccf55399141b9dca15a516e0b7cc3248f5c2cd7: Status 404 returned error can't find the container with id ffa27ad97e1afa5a0d2f9885cccf55399141b9dca15a516e0b7cc3248f5c2cd7
Jul 02 13:25:55.528168 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:55.528131    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-gk29w" event={"ID":"c99e727e-7b07-4e77-8452-ce73f33cd3f6","Type":"ContainerStarted","Data":"ffa27ad97e1afa5a0d2f9885cccf55399141b9dca15a516e0b7cc3248f5c2cd7"}
Jul 02 13:25:57.852117 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:57.852084    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:25:57.952865 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:57.952830    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:25:58.013642 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:58.013602    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:25:58.545304 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:58.545266    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-gk29w" event={"ID":"c99e727e-7b07-4e77-8452-ce73f33cd3f6","Type":"ContainerStarted","Data":"f7743257577b6d094a1bba26f97b15e036f07edfb731fe1a4081ace44fe54316"}
Jul 02 13:25:58.545521 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:58.545332    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-fd65d7d6-gk29w"
Jul 02 13:25:58.546806 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:58.546774    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-ddtp6" event={"ID":"94c22db7-55c6-4708-aba0-c2c1c9cc9576","Type":"ContainerStarted","Data":"534f11e18e90fd6847545abbbb00b2922ef3d94377aec615fac7a3bad46d8275"}
Jul 02 13:25:58.546922 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:58.546835    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-ddtp6"
Jul 02 13:25:58.562605 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:58.562544    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-fd65d7d6-gk29w" podStartSLOduration=2.342001126 podStartE2EDuration="5.562523344s" totalImagesPullingTime="3.220522218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:25:58.560656617 +0000 UTC m=+710.183449535" watchObservedRunningTime="2026-07-02 13:25:58.562523344 +0000 UTC m=+710.185316253"
Jul 02 13:25:58.575157 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:25:58.575103    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-ddtp6" podStartSLOduration=1.79571594 podStartE2EDuration="5.575088642s" totalImagesPullingTime="3.779372702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:25:58.573289914 +0000 UTC m=+710.196082822" watchObservedRunningTime="2026-07-02 13:25:58.575088642 +0000 UTC m=+710.197881585"
Jul 02 13:26:00.874043 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:00.874003    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:26:09.554055 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:09.554025    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-ddtp6"
Jul 02 13:26:09.555788 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:09.555766    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-fd65d7d6-gk29w"
Jul 02 13:26:10.394271 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:10.394235    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-tp4l8"]
Jul 02 13:26:10.397689 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:10.397671    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tp4l8"
Jul 02 13:26:10.405502 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:10.405477    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-tp4l8"]
Jul 02 13:26:10.458685 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:10.458645    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ck2c\" (UniqueName: \"kubernetes.io/projected/e68a0a1b-e0c6-412a-a601-e96804858c87-kube-api-access-8ck2c\") pod \"s3-init-tp4l8\" (UID: \"e68a0a1b-e0c6-412a-a601-e96804858c87\") " pod="kserve/s3-init-tp4l8"
Jul 02 13:26:10.458855 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:10.458691    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e68a0a1b-e0c6-412a-a601-e96804858c87-model-cache\") pod \"s3-init-tp4l8\" (UID: \"e68a0a1b-e0c6-412a-a601-e96804858c87\") " pod="kserve/s3-init-tp4l8"
Jul 02 13:26:10.559999 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:10.559948    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ck2c\" (UniqueName: \"kubernetes.io/projected/e68a0a1b-e0c6-412a-a601-e96804858c87-kube-api-access-8ck2c\") pod \"s3-init-tp4l8\" (UID: \"e68a0a1b-e0c6-412a-a601-e96804858c87\") " pod="kserve/s3-init-tp4l8"
Jul 02 13:26:10.559999 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:10.560009    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e68a0a1b-e0c6-412a-a601-e96804858c87-model-cache\") pod \"s3-init-tp4l8\" (UID: \"e68a0a1b-e0c6-412a-a601-e96804858c87\") " pod="kserve/s3-init-tp4l8"
Jul 02 13:26:10.560434 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:10.560343    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e68a0a1b-e0c6-412a-a601-e96804858c87-model-cache\") pod \"s3-init-tp4l8\" (UID: \"e68a0a1b-e0c6-412a-a601-e96804858c87\") " pod="kserve/s3-init-tp4l8"
Jul 02 13:26:10.567028 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:10.567003    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ck2c\" (UniqueName: \"kubernetes.io/projected/e68a0a1b-e0c6-412a-a601-e96804858c87-kube-api-access-8ck2c\") pod \"s3-init-tp4l8\" (UID: \"e68a0a1b-e0c6-412a-a601-e96804858c87\") " pod="kserve/s3-init-tp4l8"
Jul 02 13:26:10.715737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:10.715710    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tp4l8"
Jul 02 13:26:10.846702 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:10.846675    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-tp4l8"]
Jul 02 13:26:10.848564 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:26:10.848533    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode68a0a1b_e0c6_412a_a601_e96804858c87.slice/crio-f6958e823365c79152bf6d09d0dfb395f30816dfeae4113d89c50be30a35bc57 WatchSource:0}: Error finding container f6958e823365c79152bf6d09d0dfb395f30816dfeae4113d89c50be30a35bc57: Status 404 returned error can't find the container with id f6958e823365c79152bf6d09d0dfb395f30816dfeae4113d89c50be30a35bc57
Jul 02 13:26:11.607640 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:11.607600    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tp4l8" event={"ID":"e68a0a1b-e0c6-412a-a601-e96804858c87","Type":"ContainerStarted","Data":"f6958e823365c79152bf6d09d0dfb395f30816dfeae4113d89c50be30a35bc57"}
Jul 02 13:26:14.626517 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:14.626481    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tp4l8" event={"ID":"e68a0a1b-e0c6-412a-a601-e96804858c87","Type":"ContainerStarted","Data":"c05900cda2b8f01642122ba514fa0917243e59b82feae8d16559192b82272036"}
Jul 02 13:26:15.632875 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:15.632788    2570 generic.go:356] "Generic (PLEG): container finished" podID="e68a0a1b-e0c6-412a-a601-e96804858c87" containerID="c05900cda2b8f01642122ba514fa0917243e59b82feae8d16559192b82272036" exitCode=0
Jul 02 13:26:15.633219 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:15.632871    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tp4l8" event={"ID":"e68a0a1b-e0c6-412a-a601-e96804858c87","Type":"ContainerDied","Data":"c05900cda2b8f01642122ba514fa0917243e59b82feae8d16559192b82272036"}
Jul 02 13:26:20.663650 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:20.663613    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tp4l8" event={"ID":"e68a0a1b-e0c6-412a-a601-e96804858c87","Type":"ContainerStarted","Data":"8c4995381a8dd4d34f7a8f9c98b04f98e33ed25ed03d0603aa3d8198fd379d5d"}
Jul 02 13:26:20.679264 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:20.679199    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-tp4l8" podStartSLOduration=2.043768253 podStartE2EDuration="10.679176553s" totalImagesPullingTime="7.229314381s" totalInitContainerRuntime="1.406093919s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:26:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:26:20.676355732 +0000 UTC m=+732.299148639" watchObservedRunningTime="2026-07-02 13:26:20.679176553 +0000 UTC m=+732.301969462"
Jul 02 13:26:50.936431 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:26:50.936373    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:27:06.168935 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:06.168894    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:27:25.959174 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:25.959140    2570 generic.go:356] "Generic (PLEG): container finished" podID="e68a0a1b-e0c6-412a-a601-e96804858c87" containerID="8c4995381a8dd4d34f7a8f9c98b04f98e33ed25ed03d0603aa3d8198fd379d5d" exitCode=0
Jul 02 13:27:25.959613 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:25.959202    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tp4l8" event={"ID":"e68a0a1b-e0c6-412a-a601-e96804858c87","Type":"ContainerDied","Data":"8c4995381a8dd4d34f7a8f9c98b04f98e33ed25ed03d0603aa3d8198fd379d5d"}
Jul 02 13:27:27.092993 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:27.092970    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tp4l8"
Jul 02 13:27:27.221679 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:27.221590    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ck2c\" (UniqueName: \"kubernetes.io/projected/e68a0a1b-e0c6-412a-a601-e96804858c87-kube-api-access-8ck2c\") pod \"e68a0a1b-e0c6-412a-a601-e96804858c87\" (UID: \"e68a0a1b-e0c6-412a-a601-e96804858c87\") "
Jul 02 13:27:27.221679 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:27.221645    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e68a0a1b-e0c6-412a-a601-e96804858c87-model-cache\") pod \"e68a0a1b-e0c6-412a-a601-e96804858c87\" (UID: \"e68a0a1b-e0c6-412a-a601-e96804858c87\") "
Jul 02 13:27:27.223680 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:27.223658    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e68a0a1b-e0c6-412a-a601-e96804858c87-kube-api-access-8ck2c" (OuterVolumeSpecName: "kube-api-access-8ck2c") pod "e68a0a1b-e0c6-412a-a601-e96804858c87" (UID: "e68a0a1b-e0c6-412a-a601-e96804858c87"). InnerVolumeSpecName "kube-api-access-8ck2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:27:27.244329 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:27.244296    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e68a0a1b-e0c6-412a-a601-e96804858c87-model-cache" (OuterVolumeSpecName: "model-cache") pod "e68a0a1b-e0c6-412a-a601-e96804858c87" (UID: "e68a0a1b-e0c6-412a-a601-e96804858c87"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 13:27:27.323214 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:27.323178    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ck2c\" (UniqueName: \"kubernetes.io/projected/e68a0a1b-e0c6-412a-a601-e96804858c87-kube-api-access-8ck2c\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:27:27.323214 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:27.323208    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e68a0a1b-e0c6-412a-a601-e96804858c87-model-cache\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:27:27.970545 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:27.970519    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tp4l8"
Jul 02 13:27:27.970708 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:27.970518    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tp4l8" event={"ID":"e68a0a1b-e0c6-412a-a601-e96804858c87","Type":"ContainerDied","Data":"f6958e823365c79152bf6d09d0dfb395f30816dfeae4113d89c50be30a35bc57"}
Jul 02 13:27:27.970708 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:27.970630    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6958e823365c79152bf6d09d0dfb395f30816dfeae4113d89c50be30a35bc57"
Jul 02 13:27:28.307830 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:28.307725    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:27:28.405608 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:28.405567    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:27:39.709720 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.709623    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:27:39.756914 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.756880    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"]
Jul 02 13:27:39.757439 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.757423    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e68a0a1b-e0c6-412a-a601-e96804858c87" containerName="download-hf-model"
Jul 02 13:27:39.757505 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.757442    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e68a0a1b-e0c6-412a-a601-e96804858c87" containerName="download-hf-model"
Jul 02 13:27:39.757505 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.757460    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e68a0a1b-e0c6-412a-a601-e96804858c87" containerName="s3-init"
Jul 02 13:27:39.757505 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.757468    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e68a0a1b-e0c6-412a-a601-e96804858c87" containerName="s3-init"
Jul 02 13:27:39.757649 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.757566    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e68a0a1b-e0c6-412a-a601-e96804858c87" containerName="s3-init"
Jul 02 13:27:39.761096 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.761077    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:27:39.763154 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.763127    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 02 13:27:39.763284 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.763236    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-acf44-predictor-serving-cert\""
Jul 02 13:27:39.763284 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.763247    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 02 13:27:39.763284 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.763269    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-acf44-kube-rbac-proxy-sar-config\""
Jul 02 13:27:39.763459 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.763323    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-4xk2b\""
Jul 02 13:27:39.767812 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.767789    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"]
Jul 02 13:27:39.832418 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.832357    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-proxy-tls\") pod \"success-200-isvc-acf44-predictor-76c47c8649-p5q4f\" (UID: \"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9\") " pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:27:39.832607 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.832486    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-acf44-predictor-76c47c8649-p5q4f\" (UID: \"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9\") " pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:27:39.832607 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.832528    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjg28\" (UniqueName: \"kubernetes.io/projected/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-kube-api-access-pjg28\") pod \"success-200-isvc-acf44-predictor-76c47c8649-p5q4f\" (UID: \"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9\") " pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:27:39.886708 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.886664    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:27:39.933408 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.933351    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-proxy-tls\") pod \"success-200-isvc-acf44-predictor-76c47c8649-p5q4f\" (UID: \"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9\") " pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:27:39.933565 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.933445    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-acf44-predictor-76c47c8649-p5q4f\" (UID: \"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9\") " pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:27:39.933565 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.933475    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjg28\" (UniqueName: \"kubernetes.io/projected/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-kube-api-access-pjg28\") pod \"success-200-isvc-acf44-predictor-76c47c8649-p5q4f\" (UID: \"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9\") " pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:27:39.934156 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.934130    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-acf44-predictor-76c47c8649-p5q4f\" (UID: \"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9\") " pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:27:39.936055 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.936031    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-proxy-tls\") pod \"success-200-isvc-acf44-predictor-76c47c8649-p5q4f\" (UID: \"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9\") " pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:27:39.940886 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:39.940855    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjg28\" (UniqueName: \"kubernetes.io/projected/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-kube-api-access-pjg28\") pod \"success-200-isvc-acf44-predictor-76c47c8649-p5q4f\" (UID: \"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9\") " pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:27:40.011072 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.010988    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:27:40.059542 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.059513    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"]
Jul 02 13:27:40.064036 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.064016    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.065716 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.065693    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 02 13:27:40.065843 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.065722    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\""
Jul 02 13:27:40.065843 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.065722    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 02 13:27:40.066001 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.065988    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\""
Jul 02 13:27:40.074142 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.073494    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"]
Jul 02 13:27:40.074142 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.073909    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:27:40.115975 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.115944    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:27:40.135202 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.135123    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.135202 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.135170    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7slx\" (UniqueName: \"kubernetes.io/projected/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kube-api-access-r7slx\") pod \"isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.135407 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.135258    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.135407 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.135294    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.135407 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.135348    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.224298 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.224261    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:27:40.235800 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.235761    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.235800 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.235800    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7slx\" (UniqueName: \"kubernetes.io/projected/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kube-api-access-r7slx\") pod \"isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.236025 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.235852    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.236025 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.235876    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.236025 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:27:40.235933    2570 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-serving-cert: secret "isvc-sklearn-graph-1-predictor-serving-cert" not found
Jul 02 13:27:40.236025 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:27:40.236017    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-proxy-tls podName:e2f65642-bd7a-4bdf-90db-e1527c5f8a1b nodeName:}" failed. No retries permitted until 2026-07-02 13:27:40.735996859 +0000 UTC m=+812.358789748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-proxy-tls") pod "isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" (UID: "e2f65642-bd7a-4bdf-90db-e1527c5f8a1b") : secret "isvc-sklearn-graph-1-predictor-serving-cert" not found
Jul 02 13:27:40.236233 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.235939    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.236613 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.236592    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.236719 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.236703    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.236901 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.236878    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.242078 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.242050    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"]
Jul 02 13:27:40.245357 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.245325    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7slx\" (UniqueName: \"kubernetes.io/projected/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kube-api-access-r7slx\") pod \"isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.248575 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:27:40.248525    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c1ea2ad_39f7_425d_92c0_b44584cd5cd9.slice/crio-5844b2313ea79ab287762ac7bea6b2dd2e15ccf958bb5683e1d12485eec9b257 WatchSource:0}: Error finding container 5844b2313ea79ab287762ac7bea6b2dd2e15ccf958bb5683e1d12485eec9b257: Status 404 returned error can't find the container with id 5844b2313ea79ab287762ac7bea6b2dd2e15ccf958bb5683e1d12485eec9b257
Jul 02 13:27:40.263322 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.261084    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"]
Jul 02 13:27:40.270510 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.270486    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.272996 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.272963    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"]
Jul 02 13:27:40.273466 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.273442    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\""
Jul 02 13:27:40.274187 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.273739    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\""
Jul 02 13:27:40.325788 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.325754    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:27:40.337089 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.337037    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-p6mls\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.337267 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.337124    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t8jb\" (UniqueName: \"kubernetes.io/projected/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kube-api-access-9t8jb\") pod \"isvc-xgboost-graph-predictor-555fc69d56-p6mls\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.337267 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.337149    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-p6mls\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.337267 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.337215    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-p6mls\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.337267 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.337241    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-p6mls\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.362076 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.362039    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"]
Jul 02 13:27:40.366540 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.366520    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:40.368200 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.368178    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\""
Jul 02 13:27:40.368373 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.368253    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\""
Jul 02 13:27:40.372997 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.372972    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"]
Jul 02 13:27:40.426053 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.426020    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:27:40.438344 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.438313    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0528140d-dab1-4da1-8248-e6e511549ddf-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:40.438505 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.438350    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-p6mls\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.438505 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.438371    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-p6mls\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.438505 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.438438    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-p6mls\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.438672 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.438491    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctv89\" (UniqueName: \"kubernetes.io/projected/0528140d-dab1-4da1-8248-e6e511549ddf-kube-api-access-ctv89\") pod \"isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:40.438923 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.438543    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t8jb\" (UniqueName: \"kubernetes.io/projected/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kube-api-access-9t8jb\") pod \"isvc-xgboost-graph-predictor-555fc69d56-p6mls\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.439005 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.438956    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-p6mls\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.439005 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.438988    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0528140d-dab1-4da1-8248-e6e511549ddf-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:40.439102 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.439019    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0528140d-dab1-4da1-8248-e6e511549ddf-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:40.439102 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.439043    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0528140d-dab1-4da1-8248-e6e511549ddf-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:40.439230 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.439165    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-p6mls\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.439230 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.439165    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-p6mls\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.439326 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.439306    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-p6mls\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.441357 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.441341    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-p6mls\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.445353 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.445334    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t8jb\" (UniqueName: \"kubernetes.io/projected/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kube-api-access-9t8jb\") pod \"isvc-xgboost-graph-predictor-555fc69d56-p6mls\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.526942 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.526847    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:27:40.540097 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.540067    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0528140d-dab1-4da1-8248-e6e511549ddf-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:40.540235 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.540185    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctv89\" (UniqueName: \"kubernetes.io/projected/0528140d-dab1-4da1-8248-e6e511549ddf-kube-api-access-ctv89\") pod \"isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:40.540296 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.540229    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0528140d-dab1-4da1-8248-e6e511549ddf-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:40.540296 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.540264    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0528140d-dab1-4da1-8248-e6e511549ddf-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:40.540419 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.540314    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0528140d-dab1-4da1-8248-e6e511549ddf-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:40.540619 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:27:40.540593    2570 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-serving-cert: secret "isvc-sklearn-graph-2-predictor-serving-cert" not found
Jul 02 13:27:40.540738 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.540651    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0528140d-dab1-4da1-8248-e6e511549ddf-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:40.540738 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:27:40.540675    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0528140d-dab1-4da1-8248-e6e511549ddf-proxy-tls podName:0528140d-dab1-4da1-8248-e6e511549ddf nodeName:}" failed. No retries permitted until 2026-07-02 13:27:41.040656252 +0000 UTC m=+812.663449140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/0528140d-dab1-4da1-8248-e6e511549ddf-proxy-tls") pod "isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" (UID: "0528140d-dab1-4da1-8248-e6e511549ddf") : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found
Jul 02 13:27:40.540917 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.540901    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0528140d-dab1-4da1-8248-e6e511549ddf-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:40.540972 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.540959    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0528140d-dab1-4da1-8248-e6e511549ddf-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:40.549905 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.549884    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctv89\" (UniqueName: \"kubernetes.io/projected/0528140d-dab1-4da1-8248-e6e511549ddf-kube-api-access-ctv89\") pod \"isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:40.583790 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.583754    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:27:40.600645 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.600617    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:27:40.725660 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.725635    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"]
Jul 02 13:27:40.728286 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:27:40.728250    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaed4e2c7_8093_4b8d_bbc4_f30bbc0f20d3.slice/crio-affd118be873202e3cd3465f55d7773938f1201327394a952cd60b4a70b3a3f4 WatchSource:0}: Error finding container affd118be873202e3cd3465f55d7773938f1201327394a952cd60b4a70b3a3f4: Status 404 returned error can't find the container with id affd118be873202e3cd3465f55d7773938f1201327394a952cd60b4a70b3a3f4
Jul 02 13:27:40.742999 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.742972    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.745595 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.745569    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:40.830756 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:40.830718    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:27:41.011508 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:41.011468    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:27:41.036472 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:41.035610    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" event={"ID":"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3","Type":"ContainerStarted","Data":"75d9ea57b70c98505c02e816cf249f9e7e9e8d8e64c52264c7bc0578d0074945"}
Jul 02 13:27:41.036472 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:41.035679    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" event={"ID":"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3","Type":"ContainerStarted","Data":"affd118be873202e3cd3465f55d7773938f1201327394a952cd60b4a70b3a3f4"}
Jul 02 13:27:41.041429 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:41.040971    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" event={"ID":"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9","Type":"ContainerStarted","Data":"5844b2313ea79ab287762ac7bea6b2dd2e15ccf958bb5683e1d12485eec9b257"}
Jul 02 13:27:41.046190 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:41.046161    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0528140d-dab1-4da1-8248-e6e511549ddf-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:41.060569 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:41.060510    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0528140d-dab1-4da1-8248-e6e511549ddf-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:41.260760 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:41.260656    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"]
Jul 02 13:27:41.264990 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:27:41.264947    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2f65642_bd7a_4bdf_90db_e1527c5f8a1b.slice/crio-b76a8552ad6211abde8b0903b940bba248c826f36ff861c11104a6a75c1c2bef WatchSource:0}: Error finding container b76a8552ad6211abde8b0903b940bba248c826f36ff861c11104a6a75c1c2bef: Status 404 returned error can't find the container with id b76a8552ad6211abde8b0903b940bba248c826f36ff861c11104a6a75c1c2bef
Jul 02 13:27:41.281407 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:41.281203    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:27:41.500583 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:41.500295    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"]
Jul 02 13:27:41.508244 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:27:41.508155    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0528140d_dab1_4da1_8248_e6e511549ddf.slice/crio-730593178f68dfbd75e51585d3160e7944f16557e6517f3b79e9c12f03ab2fd2 WatchSource:0}: Error finding container 730593178f68dfbd75e51585d3160e7944f16557e6517f3b79e9c12f03ab2fd2: Status 404 returned error can't find the container with id 730593178f68dfbd75e51585d3160e7944f16557e6517f3b79e9c12f03ab2fd2
Jul 02 13:27:42.058238 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:42.057354    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" event={"ID":"0528140d-dab1-4da1-8248-e6e511549ddf","Type":"ContainerStarted","Data":"0e26f0c75f5285f520248e5be71028480e36603b9c6208edbc8a663fe1253053"}
Jul 02 13:27:42.058238 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:42.057447    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" event={"ID":"0528140d-dab1-4da1-8248-e6e511549ddf","Type":"ContainerStarted","Data":"730593178f68dfbd75e51585d3160e7944f16557e6517f3b79e9c12f03ab2fd2"}
Jul 02 13:27:42.068117 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:42.068079    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" event={"ID":"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b","Type":"ContainerStarted","Data":"287448fc2d100b5292fa83774ef992a6a6e8fc41a961314b7f3dc9fc611235be"}
Jul 02 13:27:42.068288 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:42.068153    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" event={"ID":"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b","Type":"ContainerStarted","Data":"b76a8552ad6211abde8b0903b940bba248c826f36ff861c11104a6a75c1c2bef"}
Jul 02 13:27:46.099234 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:46.099147    2570 generic.go:356] "Generic (PLEG): container finished" podID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerID="75d9ea57b70c98505c02e816cf249f9e7e9e8d8e64c52264c7bc0578d0074945" exitCode=0
Jul 02 13:27:46.099738 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:46.099222    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" event={"ID":"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3","Type":"ContainerDied","Data":"75d9ea57b70c98505c02e816cf249f9e7e9e8d8e64c52264c7bc0578d0074945"}
Jul 02 13:27:46.100904 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:46.100885    2570 generic.go:356] "Generic (PLEG): container finished" podID="0528140d-dab1-4da1-8248-e6e511549ddf" containerID="0e26f0c75f5285f520248e5be71028480e36603b9c6208edbc8a663fe1253053" exitCode=0
Jul 02 13:27:46.101008 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:46.100965    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" event={"ID":"0528140d-dab1-4da1-8248-e6e511549ddf","Type":"ContainerDied","Data":"0e26f0c75f5285f520248e5be71028480e36603b9c6208edbc8a663fe1253053"}
Jul 02 13:27:46.102700 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:46.102674    2570 generic.go:356] "Generic (PLEG): container finished" podID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerID="287448fc2d100b5292fa83774ef992a6a6e8fc41a961314b7f3dc9fc611235be" exitCode=0
Jul 02 13:27:46.102823 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:27:46.102743    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" event={"ID":"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b","Type":"ContainerDied","Data":"287448fc2d100b5292fa83774ef992a6a6e8fc41a961314b7f3dc9fc611235be"}
Jul 02 13:28:03.238340 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:03.238285    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" event={"ID":"0528140d-dab1-4da1-8248-e6e511549ddf","Type":"ContainerStarted","Data":"76f47dbd7e6838979ff431f8af24e2b3db670249f6069b4e1f3f50d2196f0451"}
Jul 02 13:28:03.240898 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:03.240760    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" event={"ID":"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b","Type":"ContainerStarted","Data":"27c83592e2014589a175d112b1ed6240de5312d8f84561944b475e377ecffced"}
Jul 02 13:28:03.242958 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:03.242935    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" event={"ID":"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9","Type":"ContainerStarted","Data":"83aceebd0e4f93ba83e723f5ffe221adada6b12979e85b23d8632918673964df"}
Jul 02 13:28:07.277680 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:07.277636    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" event={"ID":"0528140d-dab1-4da1-8248-e6e511549ddf","Type":"ContainerStarted","Data":"e45a7a2d0d06c09e51eda398c2b5021bbcbff269118ced44c495a06da6e9a4d0"}
Jul 02 13:28:07.278138 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:07.277716    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:28:07.280758 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:07.280718    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" event={"ID":"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b","Type":"ContainerStarted","Data":"496572a7fcee628f518c6c79b43c694018ca06ef5bce93b7221ff54c960eb394"}
Jul 02 13:28:07.280920 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:07.280819    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:28:07.283069 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:07.283038    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" event={"ID":"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9","Type":"ContainerStarted","Data":"0ec6757d2821f0bdc6d193d38d679e902b1ea214450cd7cc5d125cabfa32380c"}
Jul 02 13:28:07.283271 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:07.283253    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:28:07.283345 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:07.283304    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:28:07.284748 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:07.284723    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 02 13:28:07.294239 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:07.294194    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" podStartSLOduration=2.975052051 podStartE2EDuration="27.294180147s" totalImagesPullingTime="20.451958197s" totalInitContainerRuntime="3.867169899s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:27:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:28:07.292719865 +0000 UTC m=+838.915512770" watchObservedRunningTime="2026-07-02 13:28:07.294180147 +0000 UTC m=+838.916973055"
Jul 02 13:28:07.309056 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:07.309009    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" podStartSLOduration=1.990391595 podStartE2EDuration="28.308994718s" totalImagesPullingTime="26.318603123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:27:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:28:07.306955887 +0000 UTC m=+838.929748810" watchObservedRunningTime="2026-07-02 13:28:07.308994718 +0000 UTC m=+838.931787623"
Jul 02 13:28:07.324524 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:07.324472    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" podStartSLOduration=2.798313629 podStartE2EDuration="27.324457486s" totalImagesPullingTime="20.443948877s" totalInitContainerRuntime="4.08219498s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:27:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:28:07.322342914 +0000 UTC m=+838.945135822" watchObservedRunningTime="2026-07-02 13:28:07.324457486 +0000 UTC m=+838.947250389"
Jul 02 13:28:08.288705 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:08.288660    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 02 13:28:08.289265 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:08.289216    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:28:08.289326 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:08.289313    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:28:08.289928 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:08.289898    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 02 13:28:08.290043 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:08.290003    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 02 13:28:09.294626 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:09.294557    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 02 13:28:09.295111 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:09.294682    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 02 13:28:13.294304 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:13.294273    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:28:13.294786 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:13.294755    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 02 13:28:14.299903 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:14.299875    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:28:14.300604 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:14.300580    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:28:14.300723 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:14.300657    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 02 13:28:14.301140 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:14.301066    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 02 13:28:17.342124 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:17.342090    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" event={"ID":"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3","Type":"ContainerStarted","Data":"67dba92f923a15cf7baf79e361b0e8ad17f77b6f7563ad4c40f2a22a1271b8f2"}
Jul 02 13:28:17.342512 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:17.342153    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" event={"ID":"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3","Type":"ContainerStarted","Data":"40f3ac8a490d6b490eed5c21dad0bb6025bbd74bbc67c7e2ff0ed80ce5438b98"}
Jul 02 13:28:17.342512 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:17.342335    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:28:17.359772 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:17.359719    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" podStartSLOduration=2.72470441 podStartE2EDuration="37.359702451s" totalImagesPullingTime="30.535840557s" totalInitContainerRuntime="4.099157484s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:27:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:28:17.35798828 +0000 UTC m=+848.980781186" watchObservedRunningTime="2026-07-02 13:28:17.359702451 +0000 UTC m=+848.982495421"
Jul 02 13:28:18.347476 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:18.347442    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:28:18.348867 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:18.348833    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 02 13:28:19.352480 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:19.352437    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 02 13:28:23.295053 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:23.295006    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 02 13:28:24.300283 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:24.300244    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 02 13:28:24.301507 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:24.301482    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 02 13:28:24.356934 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:24.356906    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:28:24.357471 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:24.357438    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 02 13:28:33.295399 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:33.295358    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 02 13:28:34.300677 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:34.300641    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 02 13:28:34.301068 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:34.301042    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 02 13:28:34.357967 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:34.357913    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 02 13:28:43.294793 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:43.294751    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 02 13:28:44.300854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:44.300814    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 02 13:28:44.301270 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:44.300981    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 02 13:28:44.357737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:44.357697    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 02 13:28:53.295139 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:53.295099    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 02 13:28:54.300259 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:54.300222    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 02 13:28:54.301472 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:54.301445    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 02 13:28:54.357479 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:28:54.357441    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 02 13:29:03.295544 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:03.295506    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:29:04.300786 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:04.300749    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 02 13:29:04.301150 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:04.301036    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 02 13:29:04.357446 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:04.357377    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 02 13:29:08.890696 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:08.890667    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:29:08.892665 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:08.892642    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:29:08.897870 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:08.897849    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:29:08.900038 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:08.900015    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:29:14.300430 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:14.300376    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 02 13:29:14.302770 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:14.301548    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 02 13:29:14.358337 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:14.358297    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 02 13:29:17.893018 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:17.892972    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 02 13:29:19.884001 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:19.883966    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:29:21.219003 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:21.218966    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:29:24.301189 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:24.301146    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 02 13:29:24.358585 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:24.358558    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:29:27.893409 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:27.893362    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:29:29.893256 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:29.893226    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:29:30.127610 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.127574    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:29:30.238278 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.238248    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"]
Jul 02 13:29:30.238549 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.238525    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kserve-container" containerID="cri-o://83aceebd0e4f93ba83e723f5ffe221adada6b12979e85b23d8632918673964df" gracePeriod=30
Jul 02 13:29:30.238671 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.238619    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kube-rbac-proxy" containerID="cri-o://0ec6757d2821f0bdc6d193d38d679e902b1ea214450cd7cc5d125cabfa32380c" gracePeriod=30
Jul 02 13:29:30.244187 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.243752    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:29:30.274183 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.274143    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"]
Jul 02 13:29:30.278120 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.278097    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:29:30.279842 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.279820    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-383ed-predictor-serving-cert\""
Jul 02 13:29:30.279842 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.279838    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-383ed-kube-rbac-proxy-sar-config\""
Jul 02 13:29:30.289509 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.289329    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"]
Jul 02 13:29:30.314149 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.314111    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:29:30.369401 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.369354    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l952\" (UniqueName: \"kubernetes.io/projected/e7a6b0e7-1421-4980-b853-47d4df282a10-kube-api-access-5l952\") pod \"success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2\" (UID: \"e7a6b0e7-1421-4980-b853-47d4df282a10\") " pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:29:30.369565 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.369424    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e7a6b0e7-1421-4980-b853-47d4df282a10-proxy-tls\") pod \"success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2\" (UID: \"e7a6b0e7-1421-4980-b853-47d4df282a10\") " pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:29:30.369565 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.369467    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e7a6b0e7-1421-4980-b853-47d4df282a10-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2\" (UID: \"e7a6b0e7-1421-4980-b853-47d4df282a10\") " pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:29:30.393335 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.393306    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:29:30.445127 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.445090    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:29:30.470549 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.470517    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l952\" (UniqueName: \"kubernetes.io/projected/e7a6b0e7-1421-4980-b853-47d4df282a10-kube-api-access-5l952\") pod \"success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2\" (UID: \"e7a6b0e7-1421-4980-b853-47d4df282a10\") " pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:29:30.470714 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.470559    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e7a6b0e7-1421-4980-b853-47d4df282a10-proxy-tls\") pod \"success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2\" (UID: \"e7a6b0e7-1421-4980-b853-47d4df282a10\") " pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:29:30.470714 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.470594    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e7a6b0e7-1421-4980-b853-47d4df282a10-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2\" (UID: \"e7a6b0e7-1421-4980-b853-47d4df282a10\") " pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:29:30.471213 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.471195    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e7a6b0e7-1421-4980-b853-47d4df282a10-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2\" (UID: \"e7a6b0e7-1421-4980-b853-47d4df282a10\") " pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:29:30.473273 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.473252    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e7a6b0e7-1421-4980-b853-47d4df282a10-proxy-tls\") pod \"success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2\" (UID: \"e7a6b0e7-1421-4980-b853-47d4df282a10\") " pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:29:30.477222 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.477192    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l952\" (UniqueName: \"kubernetes.io/projected/e7a6b0e7-1421-4980-b853-47d4df282a10-kube-api-access-5l952\") pod \"success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2\" (UID: \"e7a6b0e7-1421-4980-b853-47d4df282a10\") " pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:29:30.546571 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.546482    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:29:30.590792 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.590758    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:29:30.706714 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.706679    2570 generic.go:356] "Generic (PLEG): container finished" podID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerID="0ec6757d2821f0bdc6d193d38d679e902b1ea214450cd7cc5d125cabfa32380c" exitCode=2
Jul 02 13:29:30.706872 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.706754    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" event={"ID":"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9","Type":"ContainerDied","Data":"0ec6757d2821f0bdc6d193d38d679e902b1ea214450cd7cc5d125cabfa32380c"}
Jul 02 13:29:30.720081 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:30.720058    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"]
Jul 02 13:29:30.721760 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:29:30.721734    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a6b0e7_1421_4980_b853_47d4df282a10.slice/crio-eec36794ba9bc3fb415cabfbb70e0eaca7c385cf306df447e55a0fe61584adab WatchSource:0}: Error finding container eec36794ba9bc3fb415cabfbb70e0eaca7c385cf306df447e55a0fe61584adab: Status 404 returned error can't find the container with id eec36794ba9bc3fb415cabfbb70e0eaca7c385cf306df447e55a0fe61584adab
Jul 02 13:29:31.716283 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:31.716244    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" event={"ID":"e7a6b0e7-1421-4980-b853-47d4df282a10","Type":"ContainerStarted","Data":"87160732730c6c0b88fd0ed07544f3b269170d31e1c823747a17c14fda93d625"}
Jul 02 13:29:31.716747 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:31.716311    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" event={"ID":"e7a6b0e7-1421-4980-b853-47d4df282a10","Type":"ContainerStarted","Data":"af805dd18f983bacffb6bfb375bb7eefe3d68919855ef3a5377d2129a3d3f640"}
Jul 02 13:29:31.716747 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:31.716341    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" event={"ID":"e7a6b0e7-1421-4980-b853-47d4df282a10","Type":"ContainerStarted","Data":"eec36794ba9bc3fb415cabfbb70e0eaca7c385cf306df447e55a0fe61584adab"}
Jul 02 13:29:31.716747 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:31.716415    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:29:31.716747 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:31.716450    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:29:31.717631 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:31.717605    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 02 13:29:31.734975 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:31.734922    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" podStartSLOduration=1.7349063519999999 podStartE2EDuration="1.734906352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:29:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:29:31.73261042 +0000 UTC m=+923.355403327" watchObservedRunningTime="2026-07-02 13:29:31.734906352 +0000 UTC m=+923.357699258"
Jul 02 13:29:32.721254 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:32.721214    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 02 13:29:33.289183 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.289136    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.31:8643/healthz\": dial tcp 10.132.0.31:8643: connect: connection refused"
Jul 02 13:29:33.295396 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.295348    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Jul 02 13:29:33.490140 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.490116    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:29:33.598136 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.598044    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-proxy-tls\") pod \"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9\" (UID: \"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9\") "
Jul 02 13:29:33.598286 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.598241    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-kube-rbac-proxy-sar-config\") pod \"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9\" (UID: \"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9\") "
Jul 02 13:29:33.598286 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.598272    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjg28\" (UniqueName: \"kubernetes.io/projected/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-kube-api-access-pjg28\") pod \"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9\" (UID: \"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9\") "
Jul 02 13:29:33.598601 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.598574    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" (UID: "0c1ea2ad-39f7-425d-92c0-b44584cd5cd9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:29:33.600285 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.600251    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" (UID: "0c1ea2ad-39f7-425d-92c0-b44584cd5cd9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:29:33.600419 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.600324    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-kube-api-access-pjg28" (OuterVolumeSpecName: "kube-api-access-pjg28") pod "0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" (UID: "0c1ea2ad-39f7-425d-92c0-b44584cd5cd9"). InnerVolumeSpecName "kube-api-access-pjg28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:29:33.699615 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.699577    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjg28\" (UniqueName: \"kubernetes.io/projected/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-kube-api-access-pjg28\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:29:33.699615 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.699607    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-proxy-tls\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:29:33.699615 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.699619    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:29:33.727832 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.727798    2570 generic.go:356] "Generic (PLEG): container finished" podID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerID="83aceebd0e4f93ba83e723f5ffe221adada6b12979e85b23d8632918673964df" exitCode=0
Jul 02 13:29:33.728228 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.727849    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"
Jul 02 13:29:33.728228 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.727885    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" event={"ID":"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9","Type":"ContainerDied","Data":"83aceebd0e4f93ba83e723f5ffe221adada6b12979e85b23d8632918673964df"}
Jul 02 13:29:33.728228 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.727963    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f" event={"ID":"0c1ea2ad-39f7-425d-92c0-b44584cd5cd9","Type":"ContainerDied","Data":"5844b2313ea79ab287762ac7bea6b2dd2e15ccf958bb5683e1d12485eec9b257"}
Jul 02 13:29:33.728228 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.727992    2570 scope.go:117] "RemoveContainer" containerID="0ec6757d2821f0bdc6d193d38d679e902b1ea214450cd7cc5d125cabfa32380c"
Jul 02 13:29:33.738669 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.738633    2570 scope.go:117] "RemoveContainer" containerID="83aceebd0e4f93ba83e723f5ffe221adada6b12979e85b23d8632918673964df"
Jul 02 13:29:33.746626 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.746609    2570 scope.go:117] "RemoveContainer" containerID="0ec6757d2821f0bdc6d193d38d679e902b1ea214450cd7cc5d125cabfa32380c"
Jul 02 13:29:33.746864 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:29:33.746847    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ec6757d2821f0bdc6d193d38d679e902b1ea214450cd7cc5d125cabfa32380c\": container with ID starting with 0ec6757d2821f0bdc6d193d38d679e902b1ea214450cd7cc5d125cabfa32380c not found: ID does not exist" containerID="0ec6757d2821f0bdc6d193d38d679e902b1ea214450cd7cc5d125cabfa32380c"
Jul 02 13:29:33.746909 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.746872    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ec6757d2821f0bdc6d193d38d679e902b1ea214450cd7cc5d125cabfa32380c"} err="failed to get container status \"0ec6757d2821f0bdc6d193d38d679e902b1ea214450cd7cc5d125cabfa32380c\": rpc error: code = NotFound desc = could not find container \"0ec6757d2821f0bdc6d193d38d679e902b1ea214450cd7cc5d125cabfa32380c\": container with ID starting with 0ec6757d2821f0bdc6d193d38d679e902b1ea214450cd7cc5d125cabfa32380c not found: ID does not exist"
Jul 02 13:29:33.746909 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.746888    2570 scope.go:117] "RemoveContainer" containerID="83aceebd0e4f93ba83e723f5ffe221adada6b12979e85b23d8632918673964df"
Jul 02 13:29:33.747114 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:29:33.747092    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83aceebd0e4f93ba83e723f5ffe221adada6b12979e85b23d8632918673964df\": container with ID starting with 83aceebd0e4f93ba83e723f5ffe221adada6b12979e85b23d8632918673964df not found: ID does not exist" containerID="83aceebd0e4f93ba83e723f5ffe221adada6b12979e85b23d8632918673964df"
Jul 02 13:29:33.747159 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.747124    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83aceebd0e4f93ba83e723f5ffe221adada6b12979e85b23d8632918673964df"} err="failed to get container status \"83aceebd0e4f93ba83e723f5ffe221adada6b12979e85b23d8632918673964df\": rpc error: code = NotFound desc = could not find container \"83aceebd0e4f93ba83e723f5ffe221adada6b12979e85b23d8632918673964df\": container with ID starting with 83aceebd0e4f93ba83e723f5ffe221adada6b12979e85b23d8632918673964df not found: ID does not exist"
Jul 02 13:29:33.748135 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.748118    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"]
Jul 02 13:29:33.751432 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:33.751410    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-acf44-predictor-76c47c8649-p5q4f"]
Jul 02 13:29:34.899133 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:34.899097    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" path="/var/lib/kubelet/pods/0c1ea2ad-39f7-425d-92c0-b44584cd5cd9/volumes"
Jul 02 13:29:37.725856 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:37.725827    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:29:37.726473 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:37.726446    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 02 13:29:47.726719 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:47.726666    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 02 13:29:57.727171 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:57.727129    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 02 13:29:59.828133 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:59.828064    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:29:59.919630 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:29:59.919594    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:30:07.726893 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:07.726851    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 02 13:30:09.946041 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:09.946005    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:30:10.006163 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.006121    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"]
Jul 02 13:30:10.006445 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.006414    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kserve-container" containerID="cri-o://40f3ac8a490d6b490eed5c21dad0bb6025bbd74bbc67c7e2ff0ed80ce5438b98" gracePeriod=30
Jul 02 13:30:10.006589 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.006458    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kube-rbac-proxy" containerID="cri-o://67dba92f923a15cf7baf79e361b0e8ad17f77b6f7563ad4c40f2a22a1271b8f2" gracePeriod=30
Jul 02 13:30:10.054377 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.054334    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"]
Jul 02 13:30:10.055277 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.054602    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kserve-container" containerID="cri-o://76f47dbd7e6838979ff431f8af24e2b3db670249f6069b4e1f3f50d2196f0451" gracePeriod=30
Jul 02 13:30:10.055277 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.054666    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kube-rbac-proxy" containerID="cri-o://e45a7a2d0d06c09e51eda398c2b5021bbcbff269118ced44c495a06da6e9a4d0" gracePeriod=30
Jul 02 13:30:10.062151 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.062124    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:30:10.079622 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.079597    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"]
Jul 02 13:30:10.080057 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.080043    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kserve-container"
Jul 02 13:30:10.080103 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.080058    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kserve-container"
Jul 02 13:30:10.080103 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.080080    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kube-rbac-proxy"
Jul 02 13:30:10.080103 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.080085    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kube-rbac-proxy"
Jul 02 13:30:10.080194 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.080145    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kube-rbac-proxy"
Jul 02 13:30:10.080194 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.080155    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c1ea2ad-39f7-425d-92c0-b44584cd5cd9" containerName="kserve-container"
Jul 02 13:30:10.084543 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.084525    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:30:10.086883 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.086840    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f369f-predictor-serving-cert\""
Jul 02 13:30:10.087011 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.086844    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f369f-kube-rbac-proxy-sar-config\""
Jul 02 13:30:10.092510 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.092206    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"]
Jul 02 13:30:10.110994 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.110961    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"]
Jul 02 13:30:10.111264 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.111236    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kserve-container" containerID="cri-o://27c83592e2014589a175d112b1ed6240de5312d8f84561944b475e377ecffced" gracePeriod=30
Jul 02 13:30:10.111357 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.111283    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kube-rbac-proxy" containerID="cri-o://496572a7fcee628f518c6c79b43c694018ca06ef5bce93b7221ff54c960eb394" gracePeriod=30
Jul 02 13:30:10.136940 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.136910    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:30:10.137488 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.137461    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/beb4e33f-3c05-4075-b592-f5951759c36a-proxy-tls\") pod \"success-200-isvc-f369f-predictor-7957879bc7-pvbxz\" (UID: \"beb4e33f-3c05-4075-b592-f5951759c36a\") " pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:30:10.137601 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.137512    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx2xb\" (UniqueName: \"kubernetes.io/projected/beb4e33f-3c05-4075-b592-f5951759c36a-kube-api-access-qx2xb\") pod \"success-200-isvc-f369f-predictor-7957879bc7-pvbxz\" (UID: \"beb4e33f-3c05-4075-b592-f5951759c36a\") " pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:30:10.137666 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.137593    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/beb4e33f-3c05-4075-b592-f5951759c36a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f369f-predictor-7957879bc7-pvbxz\" (UID: \"beb4e33f-3c05-4075-b592-f5951759c36a\") " pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:30:10.196209 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.196101    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:30:10.238371 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.238339    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/beb4e33f-3c05-4075-b592-f5951759c36a-proxy-tls\") pod \"success-200-isvc-f369f-predictor-7957879bc7-pvbxz\" (UID: \"beb4e33f-3c05-4075-b592-f5951759c36a\") " pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:30:10.238558 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.238417    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx2xb\" (UniqueName: \"kubernetes.io/projected/beb4e33f-3c05-4075-b592-f5951759c36a-kube-api-access-qx2xb\") pod \"success-200-isvc-f369f-predictor-7957879bc7-pvbxz\" (UID: \"beb4e33f-3c05-4075-b592-f5951759c36a\") " pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:30:10.238558 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.238472    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/beb4e33f-3c05-4075-b592-f5951759c36a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f369f-predictor-7957879bc7-pvbxz\" (UID: \"beb4e33f-3c05-4075-b592-f5951759c36a\") " pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:30:10.238558 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:30:10.238515    2570 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-f369f-predictor-serving-cert: secret "success-200-isvc-f369f-predictor-serving-cert" not found
Jul 02 13:30:10.238708 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:30:10.238582    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/beb4e33f-3c05-4075-b592-f5951759c36a-proxy-tls podName:beb4e33f-3c05-4075-b592-f5951759c36a nodeName:}" failed. No retries permitted until 2026-07-02 13:30:10.73856072 +0000 UTC m=+962.361353617 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/beb4e33f-3c05-4075-b592-f5951759c36a-proxy-tls") pod "success-200-isvc-f369f-predictor-7957879bc7-pvbxz" (UID: "beb4e33f-3c05-4075-b592-f5951759c36a") : secret "success-200-isvc-f369f-predictor-serving-cert" not found
Jul 02 13:30:10.239172 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.239146    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/beb4e33f-3c05-4075-b592-f5951759c36a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f369f-predictor-7957879bc7-pvbxz\" (UID: \"beb4e33f-3c05-4075-b592-f5951759c36a\") " pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:30:10.246801 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.246767    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx2xb\" (UniqueName: \"kubernetes.io/projected/beb4e33f-3c05-4075-b592-f5951759c36a-kube-api-access-qx2xb\") pod \"success-200-isvc-f369f-predictor-7957879bc7-pvbxz\" (UID: \"beb4e33f-3c05-4075-b592-f5951759c36a\") " pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:30:10.418035 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.417994    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:30:10.684139 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.684057    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:30:10.744028 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.743995    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/beb4e33f-3c05-4075-b592-f5951759c36a-proxy-tls\") pod \"success-200-isvc-f369f-predictor-7957879bc7-pvbxz\" (UID: \"beb4e33f-3c05-4075-b592-f5951759c36a\") " pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:30:10.746512 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.746489    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/beb4e33f-3c05-4075-b592-f5951759c36a-proxy-tls\") pod \"success-200-isvc-f369f-predictor-7957879bc7-pvbxz\" (UID: \"beb4e33f-3c05-4075-b592-f5951759c36a\") " pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:30:10.916521 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.916486    2570 generic.go:356] "Generic (PLEG): container finished" podID="0528140d-dab1-4da1-8248-e6e511549ddf" containerID="e45a7a2d0d06c09e51eda398c2b5021bbcbff269118ced44c495a06da6e9a4d0" exitCode=2
Jul 02 13:30:10.916687 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.916557    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" event={"ID":"0528140d-dab1-4da1-8248-e6e511549ddf","Type":"ContainerDied","Data":"e45a7a2d0d06c09e51eda398c2b5021bbcbff269118ced44c495a06da6e9a4d0"}
Jul 02 13:30:10.918463 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.918440    2570 generic.go:356] "Generic (PLEG): container finished" podID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerID="496572a7fcee628f518c6c79b43c694018ca06ef5bce93b7221ff54c960eb394" exitCode=2
Jul 02 13:30:10.918552 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.918510    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" event={"ID":"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b","Type":"ContainerDied","Data":"496572a7fcee628f518c6c79b43c694018ca06ef5bce93b7221ff54c960eb394"}
Jul 02 13:30:10.920300 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.920284    2570 generic.go:356] "Generic (PLEG): container finished" podID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerID="67dba92f923a15cf7baf79e361b0e8ad17f77b6f7563ad4c40f2a22a1271b8f2" exitCode=2
Jul 02 13:30:10.920368 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:10.920314    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" event={"ID":"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3","Type":"ContainerDied","Data":"67dba92f923a15cf7baf79e361b0e8ad17f77b6f7563ad4c40f2a22a1271b8f2"}
Jul 02 13:30:11.001335 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:11.001300    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:30:11.137822 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:11.137777    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"]
Jul 02 13:30:11.142643 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:30:11.142608    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeb4e33f_3c05_4075_b592_f5951759c36a.slice/crio-4e78111190ba6333efeff5e3535650b2f75c33703ca9204aef21472295e3f746 WatchSource:0}: Error finding container 4e78111190ba6333efeff5e3535650b2f75c33703ca9204aef21472295e3f746: Status 404 returned error can't find the container with id 4e78111190ba6333efeff5e3535650b2f75c33703ca9204aef21472295e3f746
Jul 02 13:30:11.927680 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:11.927636    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz" event={"ID":"beb4e33f-3c05-4075-b592-f5951759c36a","Type":"ContainerStarted","Data":"6d512c29bf3b7030322992dfe1c944804d0ef164c91e11415629d7420e7bd27c"}
Jul 02 13:30:11.927888 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:11.927699    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz" event={"ID":"beb4e33f-3c05-4075-b592-f5951759c36a","Type":"ContainerStarted","Data":"238578c5e807d8a684dfe63c89fe4f64eb7d22a19b2b403772236151c4ec334f"}
Jul 02 13:30:11.927888 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:11.927729    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz" event={"ID":"beb4e33f-3c05-4075-b592-f5951759c36a","Type":"ContainerStarted","Data":"4e78111190ba6333efeff5e3535650b2f75c33703ca9204aef21472295e3f746"}
Jul 02 13:30:11.927888 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:11.927788    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:30:11.945760 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:11.945701    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz" podStartSLOduration=1.9456868040000002 podStartE2EDuration="1.945686804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:30:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:30:11.944577681 +0000 UTC m=+963.567370588" watchObservedRunningTime="2026-07-02 13:30:11.945686804 +0000 UTC m=+963.568479710"
Jul 02 13:30:12.933197 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:12.933101    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:30:12.934561 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:12.934525    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz" podUID="beb4e33f-3c05-4075-b592-f5951759c36a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 02 13:30:13.941189 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:13.941128    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz" podUID="beb4e33f-3c05-4075-b592-f5951759c36a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 02 13:30:14.295062 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.295013    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.34:8643/healthz\": dial tcp 10.132.0.34:8643: connect: connection refused"
Jul 02 13:30:14.295221 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.295012    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.32:8643/healthz\": dial tcp 10.132.0.32:8643: connect: connection refused"
Jul 02 13:30:14.353041 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.352986    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.33:8643/healthz\": dial tcp 10.132.0.33:8643: connect: connection refused"
Jul 02 13:30:14.357402 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.357354    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 02 13:30:14.463060 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.463035    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:30:14.579609 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.579515    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kserve-provision-location\") pod \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") "
Jul 02 13:30:14.579609 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.579556    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-cabundle-cert\") pod \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") "
Jul 02 13:30:14.579609 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.579583    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-proxy-tls\") pod \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") "
Jul 02 13:30:14.579609 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.579613    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kube-rbac-proxy-sar-config\") pod \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") "
Jul 02 13:30:14.579953 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.579675    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t8jb\" (UniqueName: \"kubernetes.io/projected/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kube-api-access-9t8jb\") pod \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\" (UID: \"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3\") "
Jul 02 13:30:14.579953 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.579918    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" (UID: "aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 13:30:14.580037 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.579960    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" (UID: "aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:30:14.580037 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.580012    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" (UID: "aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:30:14.581791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.581767    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" (UID: "aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:30:14.581892 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.581845    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kube-api-access-9t8jb" (OuterVolumeSpecName: "kube-api-access-9t8jb") pod "aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" (UID: "aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3"). InnerVolumeSpecName "kube-api-access-9t8jb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:30:14.680650 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.680610    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t8jb\" (UniqueName: \"kubernetes.io/projected/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kube-api-access-9t8jb\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:30:14.680650 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.680643    2570 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kserve-provision-location\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:30:14.680650 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.680653    2570 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-cabundle-cert\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:30:14.680650 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.680664    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-proxy-tls\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:30:14.680911 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.680674    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:30:14.947599 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.947567    2570 generic.go:356] "Generic (PLEG): container finished" podID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerID="40f3ac8a490d6b490eed5c21dad0bb6025bbd74bbc67c7e2ff0ed80ce5438b98" exitCode=0
Jul 02 13:30:14.947961 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.947629    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"
Jul 02 13:30:14.947961 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.947651    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" event={"ID":"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3","Type":"ContainerDied","Data":"40f3ac8a490d6b490eed5c21dad0bb6025bbd74bbc67c7e2ff0ed80ce5438b98"}
Jul 02 13:30:14.947961 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.947726    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls" event={"ID":"aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3","Type":"ContainerDied","Data":"affd118be873202e3cd3465f55d7773938f1201327394a952cd60b4a70b3a3f4"}
Jul 02 13:30:14.947961 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.947764    2570 scope.go:117] "RemoveContainer" containerID="67dba92f923a15cf7baf79e361b0e8ad17f77b6f7563ad4c40f2a22a1271b8f2"
Jul 02 13:30:14.957406 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.957373    2570 scope.go:117] "RemoveContainer" containerID="40f3ac8a490d6b490eed5c21dad0bb6025bbd74bbc67c7e2ff0ed80ce5438b98"
Jul 02 13:30:14.964237 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.964209    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"]
Jul 02 13:30:14.966956 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.966935    2570 scope.go:117] "RemoveContainer" containerID="75d9ea57b70c98505c02e816cf249f9e7e9e8d8e64c52264c7bc0578d0074945"
Jul 02 13:30:14.968827 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.968808    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-p6mls"]
Jul 02 13:30:14.976987 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.976965    2570 scope.go:117] "RemoveContainer" containerID="67dba92f923a15cf7baf79e361b0e8ad17f77b6f7563ad4c40f2a22a1271b8f2"
Jul 02 13:30:14.977289 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:30:14.977268    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67dba92f923a15cf7baf79e361b0e8ad17f77b6f7563ad4c40f2a22a1271b8f2\": container with ID starting with 67dba92f923a15cf7baf79e361b0e8ad17f77b6f7563ad4c40f2a22a1271b8f2 not found: ID does not exist" containerID="67dba92f923a15cf7baf79e361b0e8ad17f77b6f7563ad4c40f2a22a1271b8f2"
Jul 02 13:30:14.977353 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.977299    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67dba92f923a15cf7baf79e361b0e8ad17f77b6f7563ad4c40f2a22a1271b8f2"} err="failed to get container status \"67dba92f923a15cf7baf79e361b0e8ad17f77b6f7563ad4c40f2a22a1271b8f2\": rpc error: code = NotFound desc = could not find container \"67dba92f923a15cf7baf79e361b0e8ad17f77b6f7563ad4c40f2a22a1271b8f2\": container with ID starting with 67dba92f923a15cf7baf79e361b0e8ad17f77b6f7563ad4c40f2a22a1271b8f2 not found: ID does not exist"
Jul 02 13:30:14.977353 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.977317    2570 scope.go:117] "RemoveContainer" containerID="40f3ac8a490d6b490eed5c21dad0bb6025bbd74bbc67c7e2ff0ed80ce5438b98"
Jul 02 13:30:14.977589 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:30:14.977572    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40f3ac8a490d6b490eed5c21dad0bb6025bbd74bbc67c7e2ff0ed80ce5438b98\": container with ID starting with 40f3ac8a490d6b490eed5c21dad0bb6025bbd74bbc67c7e2ff0ed80ce5438b98 not found: ID does not exist" containerID="40f3ac8a490d6b490eed5c21dad0bb6025bbd74bbc67c7e2ff0ed80ce5438b98"
Jul 02 13:30:14.977631 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.977597    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40f3ac8a490d6b490eed5c21dad0bb6025bbd74bbc67c7e2ff0ed80ce5438b98"} err="failed to get container status \"40f3ac8a490d6b490eed5c21dad0bb6025bbd74bbc67c7e2ff0ed80ce5438b98\": rpc error: code = NotFound desc = could not find container \"40f3ac8a490d6b490eed5c21dad0bb6025bbd74bbc67c7e2ff0ed80ce5438b98\": container with ID starting with 40f3ac8a490d6b490eed5c21dad0bb6025bbd74bbc67c7e2ff0ed80ce5438b98 not found: ID does not exist"
Jul 02 13:30:14.977631 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.977615    2570 scope.go:117] "RemoveContainer" containerID="75d9ea57b70c98505c02e816cf249f9e7e9e8d8e64c52264c7bc0578d0074945"
Jul 02 13:30:14.977828 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:30:14.977809    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75d9ea57b70c98505c02e816cf249f9e7e9e8d8e64c52264c7bc0578d0074945\": container with ID starting with 75d9ea57b70c98505c02e816cf249f9e7e9e8d8e64c52264c7bc0578d0074945 not found: ID does not exist" containerID="75d9ea57b70c98505c02e816cf249f9e7e9e8d8e64c52264c7bc0578d0074945"
Jul 02 13:30:14.977863 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:14.977834    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75d9ea57b70c98505c02e816cf249f9e7e9e8d8e64c52264c7bc0578d0074945"} err="failed to get container status \"75d9ea57b70c98505c02e816cf249f9e7e9e8d8e64c52264c7bc0578d0074945\": rpc error: code = NotFound desc = could not find container \"75d9ea57b70c98505c02e816cf249f9e7e9e8d8e64c52264c7bc0578d0074945\": container with ID starting with 75d9ea57b70c98505c02e816cf249f9e7e9e8d8e64c52264c7bc0578d0074945 not found: ID does not exist"
Jul 02 13:30:15.266038 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.266014    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:30:15.387078 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.387041    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-cabundle-cert\") pod \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") "
Jul 02 13:30:15.387260 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.387114    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7slx\" (UniqueName: \"kubernetes.io/projected/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kube-api-access-r7slx\") pod \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") "
Jul 02 13:30:15.387260 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.387137    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kube-rbac-proxy-sar-config\") pod \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") "
Jul 02 13:30:15.387260 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.387167    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kserve-provision-location\") pod \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") "
Jul 02 13:30:15.387260 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.387243    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-proxy-tls\") pod \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\" (UID: \"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b\") "
Jul 02 13:30:15.387558 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.387526    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" (UID: "e2f65642-bd7a-4bdf-90db-e1527c5f8a1b"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:30:15.387651 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.387561    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" (UID: "e2f65642-bd7a-4bdf-90db-e1527c5f8a1b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:30:15.387651 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.387617    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" (UID: "e2f65642-bd7a-4bdf-90db-e1527c5f8a1b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 13:30:15.389262 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.389240    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kube-api-access-r7slx" (OuterVolumeSpecName: "kube-api-access-r7slx") pod "e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" (UID: "e2f65642-bd7a-4bdf-90db-e1527c5f8a1b"). InnerVolumeSpecName "kube-api-access-r7slx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:30:15.389325 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.389309    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" (UID: "e2f65642-bd7a-4bdf-90db-e1527c5f8a1b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:30:15.488239 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.488204    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-proxy-tls\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:30:15.488239 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.488233    2570 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-cabundle-cert\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:30:15.488239 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.488242    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7slx\" (UniqueName: \"kubernetes.io/projected/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kube-api-access-r7slx\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:30:15.488510 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.488252    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:30:15.488510 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.488261    2570 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b-kserve-provision-location\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:30:15.955019 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.954989    2570 generic.go:356] "Generic (PLEG): container finished" podID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerID="27c83592e2014589a175d112b1ed6240de5312d8f84561944b475e377ecffced" exitCode=0
Jul 02 13:30:15.955379 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.955054    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" event={"ID":"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b","Type":"ContainerDied","Data":"27c83592e2014589a175d112b1ed6240de5312d8f84561944b475e377ecffced"}
Jul 02 13:30:15.955379 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.955060    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"
Jul 02 13:30:15.955379 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.955099    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk" event={"ID":"e2f65642-bd7a-4bdf-90db-e1527c5f8a1b","Type":"ContainerDied","Data":"b76a8552ad6211abde8b0903b940bba248c826f36ff861c11104a6a75c1c2bef"}
Jul 02 13:30:15.955379 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.955126    2570 scope.go:117] "RemoveContainer" containerID="496572a7fcee628f518c6c79b43c694018ca06ef5bce93b7221ff54c960eb394"
Jul 02 13:30:15.989079 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.989051    2570 scope.go:117] "RemoveContainer" containerID="27c83592e2014589a175d112b1ed6240de5312d8f84561944b475e377ecffced"
Jul 02 13:30:15.998975 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.998954    2570 scope.go:117] "RemoveContainer" containerID="287448fc2d100b5292fa83774ef992a6a6e8fc41a961314b7f3dc9fc611235be"
Jul 02 13:30:15.999784 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:15.999724    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"]
Jul 02 13:30:16.002671 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.002638    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6f5c857d79-k8xjk"]
Jul 02 13:30:16.008065 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.008047    2570 scope.go:117] "RemoveContainer" containerID="496572a7fcee628f518c6c79b43c694018ca06ef5bce93b7221ff54c960eb394"
Jul 02 13:30:16.008379 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:30:16.008354    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"496572a7fcee628f518c6c79b43c694018ca06ef5bce93b7221ff54c960eb394\": container with ID starting with 496572a7fcee628f518c6c79b43c694018ca06ef5bce93b7221ff54c960eb394 not found: ID does not exist" containerID="496572a7fcee628f518c6c79b43c694018ca06ef5bce93b7221ff54c960eb394"
Jul 02 13:30:16.008457 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.008411    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"496572a7fcee628f518c6c79b43c694018ca06ef5bce93b7221ff54c960eb394"} err="failed to get container status \"496572a7fcee628f518c6c79b43c694018ca06ef5bce93b7221ff54c960eb394\": rpc error: code = NotFound desc = could not find container \"496572a7fcee628f518c6c79b43c694018ca06ef5bce93b7221ff54c960eb394\": container with ID starting with 496572a7fcee628f518c6c79b43c694018ca06ef5bce93b7221ff54c960eb394 not found: ID does not exist"
Jul 02 13:30:16.008457 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.008439    2570 scope.go:117] "RemoveContainer" containerID="27c83592e2014589a175d112b1ed6240de5312d8f84561944b475e377ecffced"
Jul 02 13:30:16.008722 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:30:16.008703    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27c83592e2014589a175d112b1ed6240de5312d8f84561944b475e377ecffced\": container with ID starting with 27c83592e2014589a175d112b1ed6240de5312d8f84561944b475e377ecffced not found: ID does not exist" containerID="27c83592e2014589a175d112b1ed6240de5312d8f84561944b475e377ecffced"
Jul 02 13:30:16.008768 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.008728    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27c83592e2014589a175d112b1ed6240de5312d8f84561944b475e377ecffced"} err="failed to get container status \"27c83592e2014589a175d112b1ed6240de5312d8f84561944b475e377ecffced\": rpc error: code = NotFound desc = could not find container \"27c83592e2014589a175d112b1ed6240de5312d8f84561944b475e377ecffced\": container with ID starting with 27c83592e2014589a175d112b1ed6240de5312d8f84561944b475e377ecffced not found: ID does not exist"
Jul 02 13:30:16.008768 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.008745    2570 scope.go:117] "RemoveContainer" containerID="287448fc2d100b5292fa83774ef992a6a6e8fc41a961314b7f3dc9fc611235be"
Jul 02 13:30:16.008995 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:30:16.008978    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"287448fc2d100b5292fa83774ef992a6a6e8fc41a961314b7f3dc9fc611235be\": container with ID starting with 287448fc2d100b5292fa83774ef992a6a6e8fc41a961314b7f3dc9fc611235be not found: ID does not exist" containerID="287448fc2d100b5292fa83774ef992a6a6e8fc41a961314b7f3dc9fc611235be"
Jul 02 13:30:16.009051 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.009003    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"287448fc2d100b5292fa83774ef992a6a6e8fc41a961314b7f3dc9fc611235be"} err="failed to get container status \"287448fc2d100b5292fa83774ef992a6a6e8fc41a961314b7f3dc9fc611235be\": rpc error: code = NotFound desc = could not find container \"287448fc2d100b5292fa83774ef992a6a6e8fc41a961314b7f3dc9fc611235be\": container with ID starting with 287448fc2d100b5292fa83774ef992a6a6e8fc41a961314b7f3dc9fc611235be not found: ID does not exist"
Jul 02 13:30:16.110853 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.110828    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:30:16.195051 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.195013    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0528140d-dab1-4da1-8248-e6e511549ddf-kube-rbac-proxy-sar-config\") pod \"0528140d-dab1-4da1-8248-e6e511549ddf\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") "
Jul 02 13:30:16.195229 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.195063    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0528140d-dab1-4da1-8248-e6e511549ddf-cabundle-cert\") pod \"0528140d-dab1-4da1-8248-e6e511549ddf\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") "
Jul 02 13:30:16.195229 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.195102    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0528140d-dab1-4da1-8248-e6e511549ddf-proxy-tls\") pod \"0528140d-dab1-4da1-8248-e6e511549ddf\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") "
Jul 02 13:30:16.195229 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.195149    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0528140d-dab1-4da1-8248-e6e511549ddf-kserve-provision-location\") pod \"0528140d-dab1-4da1-8248-e6e511549ddf\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") "
Jul 02 13:30:16.195229 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.195194    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctv89\" (UniqueName: \"kubernetes.io/projected/0528140d-dab1-4da1-8248-e6e511549ddf-kube-api-access-ctv89\") pod \"0528140d-dab1-4da1-8248-e6e511549ddf\" (UID: \"0528140d-dab1-4da1-8248-e6e511549ddf\") "
Jul 02 13:30:16.195602 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.195567    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0528140d-dab1-4da1-8248-e6e511549ddf-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "0528140d-dab1-4da1-8248-e6e511549ddf" (UID: "0528140d-dab1-4da1-8248-e6e511549ddf"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:30:16.195602 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.195577    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0528140d-dab1-4da1-8248-e6e511549ddf-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "0528140d-dab1-4da1-8248-e6e511549ddf" (UID: "0528140d-dab1-4da1-8248-e6e511549ddf"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:30:16.195778 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.195580    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0528140d-dab1-4da1-8248-e6e511549ddf-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "0528140d-dab1-4da1-8248-e6e511549ddf" (UID: "0528140d-dab1-4da1-8248-e6e511549ddf"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 13:30:16.197358 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.197333    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0528140d-dab1-4da1-8248-e6e511549ddf-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0528140d-dab1-4da1-8248-e6e511549ddf" (UID: "0528140d-dab1-4da1-8248-e6e511549ddf"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:30:16.197541 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.197401    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0528140d-dab1-4da1-8248-e6e511549ddf-kube-api-access-ctv89" (OuterVolumeSpecName: "kube-api-access-ctv89") pod "0528140d-dab1-4da1-8248-e6e511549ddf" (UID: "0528140d-dab1-4da1-8248-e6e511549ddf"). InnerVolumeSpecName "kube-api-access-ctv89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:30:16.296221 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.296185    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctv89\" (UniqueName: \"kubernetes.io/projected/0528140d-dab1-4da1-8248-e6e511549ddf-kube-api-access-ctv89\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:30:16.296221 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.296218    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0528140d-dab1-4da1-8248-e6e511549ddf-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:30:16.296451 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.296227    2570 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0528140d-dab1-4da1-8248-e6e511549ddf-cabundle-cert\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:30:16.296451 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.296249    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0528140d-dab1-4da1-8248-e6e511549ddf-proxy-tls\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:30:16.296451 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.296261    2570 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0528140d-dab1-4da1-8248-e6e511549ddf-kserve-provision-location\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:30:16.899067 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.899019    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" path="/var/lib/kubelet/pods/aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3/volumes"
Jul 02 13:30:16.899560 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.899546    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" path="/var/lib/kubelet/pods/e2f65642-bd7a-4bdf-90db-e1527c5f8a1b/volumes"
Jul 02 13:30:16.963560 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.963525    2570 generic.go:356] "Generic (PLEG): container finished" podID="0528140d-dab1-4da1-8248-e6e511549ddf" containerID="76f47dbd7e6838979ff431f8af24e2b3db670249f6069b4e1f3f50d2196f0451" exitCode=0
Jul 02 13:30:16.963991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.963577    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"
Jul 02 13:30:16.963991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.963597    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" event={"ID":"0528140d-dab1-4da1-8248-e6e511549ddf","Type":"ContainerDied","Data":"76f47dbd7e6838979ff431f8af24e2b3db670249f6069b4e1f3f50d2196f0451"}
Jul 02 13:30:16.963991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.963653    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn" event={"ID":"0528140d-dab1-4da1-8248-e6e511549ddf","Type":"ContainerDied","Data":"730593178f68dfbd75e51585d3160e7944f16557e6517f3b79e9c12f03ab2fd2"}
Jul 02 13:30:16.963991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.963689    2570 scope.go:117] "RemoveContainer" containerID="e45a7a2d0d06c09e51eda398c2b5021bbcbff269118ced44c495a06da6e9a4d0"
Jul 02 13:30:16.972788 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.972764    2570 scope.go:117] "RemoveContainer" containerID="76f47dbd7e6838979ff431f8af24e2b3db670249f6069b4e1f3f50d2196f0451"
Jul 02 13:30:16.979289 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.979264    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"]
Jul 02 13:30:16.981288 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.981271    2570 scope.go:117] "RemoveContainer" containerID="0e26f0c75f5285f520248e5be71028480e36603b9c6208edbc8a663fe1253053"
Jul 02 13:30:16.984652 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.984629    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6fb9776d89-b7dzn"]
Jul 02 13:30:16.989121 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.989101    2570 scope.go:117] "RemoveContainer" containerID="e45a7a2d0d06c09e51eda398c2b5021bbcbff269118ced44c495a06da6e9a4d0"
Jul 02 13:30:16.989377 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:30:16.989358    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e45a7a2d0d06c09e51eda398c2b5021bbcbff269118ced44c495a06da6e9a4d0\": container with ID starting with e45a7a2d0d06c09e51eda398c2b5021bbcbff269118ced44c495a06da6e9a4d0 not found: ID does not exist" containerID="e45a7a2d0d06c09e51eda398c2b5021bbcbff269118ced44c495a06da6e9a4d0"
Jul 02 13:30:16.989476 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.989403    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e45a7a2d0d06c09e51eda398c2b5021bbcbff269118ced44c495a06da6e9a4d0"} err="failed to get container status \"e45a7a2d0d06c09e51eda398c2b5021bbcbff269118ced44c495a06da6e9a4d0\": rpc error: code = NotFound desc = could not find container \"e45a7a2d0d06c09e51eda398c2b5021bbcbff269118ced44c495a06da6e9a4d0\": container with ID starting with e45a7a2d0d06c09e51eda398c2b5021bbcbff269118ced44c495a06da6e9a4d0 not found: ID does not exist"
Jul 02 13:30:16.989476 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.989430    2570 scope.go:117] "RemoveContainer" containerID="76f47dbd7e6838979ff431f8af24e2b3db670249f6069b4e1f3f50d2196f0451"
Jul 02 13:30:16.989703 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:30:16.989685    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76f47dbd7e6838979ff431f8af24e2b3db670249f6069b4e1f3f50d2196f0451\": container with ID starting with 76f47dbd7e6838979ff431f8af24e2b3db670249f6069b4e1f3f50d2196f0451 not found: ID does not exist" containerID="76f47dbd7e6838979ff431f8af24e2b3db670249f6069b4e1f3f50d2196f0451"
Jul 02 13:30:16.989749 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.989709    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76f47dbd7e6838979ff431f8af24e2b3db670249f6069b4e1f3f50d2196f0451"} err="failed to get container status \"76f47dbd7e6838979ff431f8af24e2b3db670249f6069b4e1f3f50d2196f0451\": rpc error: code = NotFound desc = could not find container \"76f47dbd7e6838979ff431f8af24e2b3db670249f6069b4e1f3f50d2196f0451\": container with ID starting with 76f47dbd7e6838979ff431f8af24e2b3db670249f6069b4e1f3f50d2196f0451 not found: ID does not exist"
Jul 02 13:30:16.989749 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.989725    2570 scope.go:117] "RemoveContainer" containerID="0e26f0c75f5285f520248e5be71028480e36603b9c6208edbc8a663fe1253053"
Jul 02 13:30:16.990000 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:30:16.989982    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e26f0c75f5285f520248e5be71028480e36603b9c6208edbc8a663fe1253053\": container with ID starting with 0e26f0c75f5285f520248e5be71028480e36603b9c6208edbc8a663fe1253053 not found: ID does not exist" containerID="0e26f0c75f5285f520248e5be71028480e36603b9c6208edbc8a663fe1253053"
Jul 02 13:30:16.990052 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:16.990004    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e26f0c75f5285f520248e5be71028480e36603b9c6208edbc8a663fe1253053"} err="failed to get container status \"0e26f0c75f5285f520248e5be71028480e36603b9c6208edbc8a663fe1253053\": rpc error: code = NotFound desc = could not find container \"0e26f0c75f5285f520248e5be71028480e36603b9c6208edbc8a663fe1253053\": container with ID starting with 0e26f0c75f5285f520248e5be71028480e36603b9c6208edbc8a663fe1253053 not found: ID does not exist"
Jul 02 13:30:17.727591 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:17.727557    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:30:18.899130 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:18.899090    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" path="/var/lib/kubelet/pods/0528140d-dab1-4da1-8248-e6e511549ddf/volumes"
Jul 02 13:30:18.945307 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:18.945275    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:30:18.945748 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:18.945721    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz" podUID="beb4e33f-3c05-4075-b592-f5951759c36a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 02 13:30:28.946652 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:28.946611    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz" podUID="beb4e33f-3c05-4075-b592-f5951759c36a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 02 13:30:30.326543 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:30.326505    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:30:30.556822 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:30.556784    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:30:38.946739 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:38.946651    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz" podUID="beb4e33f-3c05-4075-b592-f5951759c36a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 02 13:30:48.946686 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:48.946649    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz" podUID="beb4e33f-3c05-4075-b592-f5951759c36a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 02 13:30:58.947262 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:30:58.947231    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:31:10.104922 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:31:10.104885    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:31:10.237439 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:31:10.237362    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:31:39.458513 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:31:39.458474    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:33:07.062833 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:33:07.062797    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:33:58.846455 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:33:58.846352    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:34:08.923083 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:34:08.923053    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:34:08.925934 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:34:08.925902    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:34:08.928649 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:34:08.928629    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:34:08.931841 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:34:08.931822    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:35:43.136879 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:35:43.136840    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:36:24.029683 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:36:24.029643    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:36:24.075399 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:36:24.075353    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:38:45.132064 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.132029    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"]
Jul 02 13:38:45.132648 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.132351    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerName="kserve-container" containerID="cri-o://af805dd18f983bacffb6bfb375bb7eefe3d68919855ef3a5377d2129a3d3f640" gracePeriod=30
Jul 02 13:38:45.132648 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.132485    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerName="kube-rbac-proxy" containerID="cri-o://87160732730c6c0b88fd0ed07544f3b269170d31e1c823747a17c14fda93d625" gracePeriod=30
Jul 02 13:38:45.184770 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.184739    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"]
Jul 02 13:38:45.185147 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185135    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="storage-initializer"
Jul 02 13:38:45.185190 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185148    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="storage-initializer"
Jul 02 13:38:45.185190 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185157    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kserve-container"
Jul 02 13:38:45.185190 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185162    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kserve-container"
Jul 02 13:38:45.185190 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185172    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kserve-container"
Jul 02 13:38:45.185190 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185177    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kserve-container"
Jul 02 13:38:45.185190 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185187    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="storage-initializer"
Jul 02 13:38:45.185190 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185192    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="storage-initializer"
Jul 02 13:38:45.185416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185198    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kube-rbac-proxy"
Jul 02 13:38:45.185416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185203    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kube-rbac-proxy"
Jul 02 13:38:45.185416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185209    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kube-rbac-proxy"
Jul 02 13:38:45.185416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185214    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kube-rbac-proxy"
Jul 02 13:38:45.185416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185222    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kube-rbac-proxy"
Jul 02 13:38:45.185416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185227    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kube-rbac-proxy"
Jul 02 13:38:45.185416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185233    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kserve-container"
Jul 02 13:38:45.185416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185239    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kserve-container"
Jul 02 13:38:45.185416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185245    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="storage-initializer"
Jul 02 13:38:45.185416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185250    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="storage-initializer"
Jul 02 13:38:45.185416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185303    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kube-rbac-proxy"
Jul 02 13:38:45.185416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185311    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kserve-container"
Jul 02 13:38:45.185416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185317    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0528140d-dab1-4da1-8248-e6e511549ddf" containerName="kserve-container"
Jul 02 13:38:45.185416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185323    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kube-rbac-proxy"
Jul 02 13:38:45.185416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185329    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="aed4e2c7-8093-4b8d-bbc4-f30bbc0f20d3" containerName="kube-rbac-proxy"
Jul 02 13:38:45.185416 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.185335    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2f65642-bd7a-4bdf-90db-e1527c5f8a1b" containerName="kserve-container"
Jul 02 13:38:45.188606 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.188578    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:38:45.190317 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.190293    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-6c5e8-kube-rbac-proxy-sar-config\""
Jul 02 13:38:45.190454 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.190294    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-6c5e8-predictor-serving-cert\""
Jul 02 13:38:45.199950 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.199924    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"]
Jul 02 13:38:45.332245 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.332209    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/328ec472-00f3-45f9-ab8b-9ae649b29ee8-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6c5e8-predictor-866c9df647-kc54f\" (UID: \"328ec472-00f3-45f9-ab8b-9ae649b29ee8\") " pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:38:45.332465 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.332264    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/328ec472-00f3-45f9-ab8b-9ae649b29ee8-proxy-tls\") pod \"success-200-isvc-6c5e8-predictor-866c9df647-kc54f\" (UID: \"328ec472-00f3-45f9-ab8b-9ae649b29ee8\") " pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:38:45.332465 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.332362    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk7wt\" (UniqueName: \"kubernetes.io/projected/328ec472-00f3-45f9-ab8b-9ae649b29ee8-kube-api-access-tk7wt\") pod \"success-200-isvc-6c5e8-predictor-866c9df647-kc54f\" (UID: \"328ec472-00f3-45f9-ab8b-9ae649b29ee8\") " pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:38:45.374061 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.374016    2570 generic.go:356] "Generic (PLEG): container finished" podID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerID="87160732730c6c0b88fd0ed07544f3b269170d31e1c823747a17c14fda93d625" exitCode=2
Jul 02 13:38:45.374240 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.374075    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" event={"ID":"e7a6b0e7-1421-4980-b853-47d4df282a10","Type":"ContainerDied","Data":"87160732730c6c0b88fd0ed07544f3b269170d31e1c823747a17c14fda93d625"}
Jul 02 13:38:45.433399 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.433297    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk7wt\" (UniqueName: \"kubernetes.io/projected/328ec472-00f3-45f9-ab8b-9ae649b29ee8-kube-api-access-tk7wt\") pod \"success-200-isvc-6c5e8-predictor-866c9df647-kc54f\" (UID: \"328ec472-00f3-45f9-ab8b-9ae649b29ee8\") " pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:38:45.433563 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.433434    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/328ec472-00f3-45f9-ab8b-9ae649b29ee8-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6c5e8-predictor-866c9df647-kc54f\" (UID: \"328ec472-00f3-45f9-ab8b-9ae649b29ee8\") " pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:38:45.433563 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.433480    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/328ec472-00f3-45f9-ab8b-9ae649b29ee8-proxy-tls\") pod \"success-200-isvc-6c5e8-predictor-866c9df647-kc54f\" (UID: \"328ec472-00f3-45f9-ab8b-9ae649b29ee8\") " pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:38:45.433692 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:38:45.433600    2570 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-serving-cert: secret "success-200-isvc-6c5e8-predictor-serving-cert" not found
Jul 02 13:38:45.433692 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:38:45.433669    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328ec472-00f3-45f9-ab8b-9ae649b29ee8-proxy-tls podName:328ec472-00f3-45f9-ab8b-9ae649b29ee8 nodeName:}" failed. No retries permitted until 2026-07-02 13:38:45.933649714 +0000 UTC m=+1477.556442611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/328ec472-00f3-45f9-ab8b-9ae649b29ee8-proxy-tls") pod "success-200-isvc-6c5e8-predictor-866c9df647-kc54f" (UID: "328ec472-00f3-45f9-ab8b-9ae649b29ee8") : secret "success-200-isvc-6c5e8-predictor-serving-cert" not found
Jul 02 13:38:45.434203 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.434176    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/328ec472-00f3-45f9-ab8b-9ae649b29ee8-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6c5e8-predictor-866c9df647-kc54f\" (UID: \"328ec472-00f3-45f9-ab8b-9ae649b29ee8\") " pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:38:45.441897 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.441868    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk7wt\" (UniqueName: \"kubernetes.io/projected/328ec472-00f3-45f9-ab8b-9ae649b29ee8-kube-api-access-tk7wt\") pod \"success-200-isvc-6c5e8-predictor-866c9df647-kc54f\" (UID: \"328ec472-00f3-45f9-ab8b-9ae649b29ee8\") " pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:38:45.827778 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.827747    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:38:45.913228 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.913197    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:38:45.938753 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.938722    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/328ec472-00f3-45f9-ab8b-9ae649b29ee8-proxy-tls\") pod \"success-200-isvc-6c5e8-predictor-866c9df647-kc54f\" (UID: \"328ec472-00f3-45f9-ab8b-9ae649b29ee8\") " pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:38:45.941095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:45.941068    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/328ec472-00f3-45f9-ab8b-9ae649b29ee8-proxy-tls\") pod \"success-200-isvc-6c5e8-predictor-866c9df647-kc54f\" (UID: \"328ec472-00f3-45f9-ab8b-9ae649b29ee8\") " pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:38:46.101405 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:46.101290    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:38:46.231196 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:46.231168    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"]
Jul 02 13:38:46.233425 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:38:46.233398    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod328ec472_00f3_45f9_ab8b_9ae649b29ee8.slice/crio-688e35cbd65297de40a8e812cd0fefa20404cf4665166c973807c2c04b13a5b3 WatchSource:0}: Error finding container 688e35cbd65297de40a8e812cd0fefa20404cf4665166c973807c2c04b13a5b3: Status 404 returned error can't find the container with id 688e35cbd65297de40a8e812cd0fefa20404cf4665166c973807c2c04b13a5b3
Jul 02 13:38:46.380000 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:46.379971    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f" event={"ID":"328ec472-00f3-45f9-ab8b-9ae649b29ee8","Type":"ContainerStarted","Data":"8a091393383977a3f361c4e8039330c52c74a7df90c4eeebdea90efb69a92f0b"}
Jul 02 13:38:46.380147 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:46.380031    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f" event={"ID":"328ec472-00f3-45f9-ab8b-9ae649b29ee8","Type":"ContainerStarted","Data":"5112e33ac831bbaea927e5543fbe02be2181899f82225f432413fbbeede6077a"}
Jul 02 13:38:46.380147 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:46.380053    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f" event={"ID":"328ec472-00f3-45f9-ab8b-9ae649b29ee8","Type":"ContainerStarted","Data":"688e35cbd65297de40a8e812cd0fefa20404cf4665166c973807c2c04b13a5b3"}
Jul 02 13:38:46.380147 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:46.380077    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:38:46.380147 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:46.380106    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:38:46.381647 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:46.381619    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f" podUID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 02 13:38:46.397092 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:46.397039    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f" podStartSLOduration=1.3970223339999999 podStartE2EDuration="1.397022334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:38:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:38:46.395220444 +0000 UTC m=+1478.018013350" watchObservedRunningTime="2026-07-02 13:38:46.397022334 +0000 UTC m=+1478.019815239"
Jul 02 13:38:47.384426 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:47.384367    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f" podUID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 02 13:38:47.721679 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:47.721637    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.35:8643/healthz\": dial tcp 10.132.0.35:8643: connect: connection refused"
Jul 02 13:38:47.727098 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:47.727069    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 02 13:38:48.285326 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.285302    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:38:48.361463 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.361427    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e7a6b0e7-1421-4980-b853-47d4df282a10-kube-rbac-proxy-sar-config\") pod \"e7a6b0e7-1421-4980-b853-47d4df282a10\" (UID: \"e7a6b0e7-1421-4980-b853-47d4df282a10\") "
Jul 02 13:38:48.361658 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.361514    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l952\" (UniqueName: \"kubernetes.io/projected/e7a6b0e7-1421-4980-b853-47d4df282a10-kube-api-access-5l952\") pod \"e7a6b0e7-1421-4980-b853-47d4df282a10\" (UID: \"e7a6b0e7-1421-4980-b853-47d4df282a10\") "
Jul 02 13:38:48.361658 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.361600    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e7a6b0e7-1421-4980-b853-47d4df282a10-proxy-tls\") pod \"e7a6b0e7-1421-4980-b853-47d4df282a10\" (UID: \"e7a6b0e7-1421-4980-b853-47d4df282a10\") "
Jul 02 13:38:48.361901 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.361874    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7a6b0e7-1421-4980-b853-47d4df282a10-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e7a6b0e7-1421-4980-b853-47d4df282a10" (UID: "e7a6b0e7-1421-4980-b853-47d4df282a10"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:38:48.363724 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.363698    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7a6b0e7-1421-4980-b853-47d4df282a10-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e7a6b0e7-1421-4980-b853-47d4df282a10" (UID: "e7a6b0e7-1421-4980-b853-47d4df282a10"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:38:48.363826 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.363718    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7a6b0e7-1421-4980-b853-47d4df282a10-kube-api-access-5l952" (OuterVolumeSpecName: "kube-api-access-5l952") pod "e7a6b0e7-1421-4980-b853-47d4df282a10" (UID: "e7a6b0e7-1421-4980-b853-47d4df282a10"). InnerVolumeSpecName "kube-api-access-5l952". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:38:48.390607 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.390521    2570 generic.go:356] "Generic (PLEG): container finished" podID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerID="af805dd18f983bacffb6bfb375bb7eefe3d68919855ef3a5377d2129a3d3f640" exitCode=0
Jul 02 13:38:48.390607 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.390569    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"
Jul 02 13:38:48.390607 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.390577    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" event={"ID":"e7a6b0e7-1421-4980-b853-47d4df282a10","Type":"ContainerDied","Data":"af805dd18f983bacffb6bfb375bb7eefe3d68919855ef3a5377d2129a3d3f640"}
Jul 02 13:38:48.391041 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.390629    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2" event={"ID":"e7a6b0e7-1421-4980-b853-47d4df282a10","Type":"ContainerDied","Data":"eec36794ba9bc3fb415cabfbb70e0eaca7c385cf306df447e55a0fe61584adab"}
Jul 02 13:38:48.391041 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.390660    2570 scope.go:117] "RemoveContainer" containerID="87160732730c6c0b88fd0ed07544f3b269170d31e1c823747a17c14fda93d625"
Jul 02 13:38:48.400096 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.400071    2570 scope.go:117] "RemoveContainer" containerID="af805dd18f983bacffb6bfb375bb7eefe3d68919855ef3a5377d2129a3d3f640"
Jul 02 13:38:48.407686 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.407656    2570 scope.go:117] "RemoveContainer" containerID="87160732730c6c0b88fd0ed07544f3b269170d31e1c823747a17c14fda93d625"
Jul 02 13:38:48.407999 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:38:48.407973    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87160732730c6c0b88fd0ed07544f3b269170d31e1c823747a17c14fda93d625\": container with ID starting with 87160732730c6c0b88fd0ed07544f3b269170d31e1c823747a17c14fda93d625 not found: ID does not exist" containerID="87160732730c6c0b88fd0ed07544f3b269170d31e1c823747a17c14fda93d625"
Jul 02 13:38:48.408089 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.408014    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87160732730c6c0b88fd0ed07544f3b269170d31e1c823747a17c14fda93d625"} err="failed to get container status \"87160732730c6c0b88fd0ed07544f3b269170d31e1c823747a17c14fda93d625\": rpc error: code = NotFound desc = could not find container \"87160732730c6c0b88fd0ed07544f3b269170d31e1c823747a17c14fda93d625\": container with ID starting with 87160732730c6c0b88fd0ed07544f3b269170d31e1c823747a17c14fda93d625 not found: ID does not exist"
Jul 02 13:38:48.408089 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.408034    2570 scope.go:117] "RemoveContainer" containerID="af805dd18f983bacffb6bfb375bb7eefe3d68919855ef3a5377d2129a3d3f640"
Jul 02 13:38:48.408361 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:38:48.408334    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af805dd18f983bacffb6bfb375bb7eefe3d68919855ef3a5377d2129a3d3f640\": container with ID starting with af805dd18f983bacffb6bfb375bb7eefe3d68919855ef3a5377d2129a3d3f640 not found: ID does not exist" containerID="af805dd18f983bacffb6bfb375bb7eefe3d68919855ef3a5377d2129a3d3f640"
Jul 02 13:38:48.408496 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.408369    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af805dd18f983bacffb6bfb375bb7eefe3d68919855ef3a5377d2129a3d3f640"} err="failed to get container status \"af805dd18f983bacffb6bfb375bb7eefe3d68919855ef3a5377d2129a3d3f640\": rpc error: code = NotFound desc = could not find container \"af805dd18f983bacffb6bfb375bb7eefe3d68919855ef3a5377d2129a3d3f640\": container with ID starting with af805dd18f983bacffb6bfb375bb7eefe3d68919855ef3a5377d2129a3d3f640 not found: ID does not exist"
Jul 02 13:38:48.410420 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.410378    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"]
Jul 02 13:38:48.414171 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.414151    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-383ed-predictor-55fbb9d4ff-mc9t2"]
Jul 02 13:38:48.462593 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.462561    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e7a6b0e7-1421-4980-b853-47d4df282a10-proxy-tls\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:38:48.462593 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.462586    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e7a6b0e7-1421-4980-b853-47d4df282a10-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:38:48.462593 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.462596    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l952\" (UniqueName: \"kubernetes.io/projected/e7a6b0e7-1421-4980-b853-47d4df282a10-kube-api-access-5l952\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:38:48.897876 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:48.897843    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" path="/var/lib/kubelet/pods/e7a6b0e7-1421-4980-b853-47d4df282a10/volumes"
Jul 02 13:38:52.389523 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:52.389494    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:38:52.390088 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:38:52.390062    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f" podUID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 02 13:39:02.390339 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:02.390296    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f" podUID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 02 13:39:08.954424 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:08.954378    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:39:08.956757 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:08.956734    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:39:08.960102 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:08.960079    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:39:08.962784 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:08.962765    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:39:12.390434 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:12.390370    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f" podUID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 02 13:39:22.390567 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:22.390531    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f" podUID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 02 13:39:24.792026 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.791958    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:39:24.900719 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.900687    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"]
Jul 02 13:39:24.901012 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.900982    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz" podUID="beb4e33f-3c05-4075-b592-f5951759c36a" containerName="kserve-container" containerID="cri-o://238578c5e807d8a684dfe63c89fe4f64eb7d22a19b2b403772236151c4ec334f" gracePeriod=30
Jul 02 13:39:24.901112 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.900984    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz" podUID="beb4e33f-3c05-4075-b592-f5951759c36a" containerName="kube-rbac-proxy" containerID="cri-o://6d512c29bf3b7030322992dfe1c944804d0ef164c91e11415629d7420e7bd27c" gracePeriod=30
Jul 02 13:39:24.907473 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.907449    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"]
Jul 02 13:39:24.908038 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.908023    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerName="kserve-container"
Jul 02 13:39:24.908109 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.908041    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerName="kserve-container"
Jul 02 13:39:24.908109 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.908065    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerName="kube-rbac-proxy"
Jul 02 13:39:24.908109 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.908074    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerName="kube-rbac-proxy"
Jul 02 13:39:24.908214 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.908165    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerName="kserve-container"
Jul 02 13:39:24.908214 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.908185    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7a6b0e7-1421-4980-b853-47d4df282a10" containerName="kube-rbac-proxy"
Jul 02 13:39:24.911854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.911836    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:39:24.913514 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.913487    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2bc9c-predictor-serving-cert\""
Jul 02 13:39:24.913514 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.913502    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2bc9c-kube-rbac-proxy-sar-config\""
Jul 02 13:39:24.920635 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.920596    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"]
Jul 02 13:39:24.967143 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.967111    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwd2x\" (UniqueName: \"kubernetes.io/projected/b32c919d-6838-47bd-ad43-56804a8f3937-kube-api-access-rwd2x\") pod \"success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl\" (UID: \"b32c919d-6838-47bd-ad43-56804a8f3937\") " pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:39:24.967337 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.967208    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b32c919d-6838-47bd-ad43-56804a8f3937-proxy-tls\") pod \"success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl\" (UID: \"b32c919d-6838-47bd-ad43-56804a8f3937\") " pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:39:24.967337 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:24.967259    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b32c919d-6838-47bd-ad43-56804a8f3937-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl\" (UID: \"b32c919d-6838-47bd-ad43-56804a8f3937\") " pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:39:25.068039 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:25.067948    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b32c919d-6838-47bd-ad43-56804a8f3937-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl\" (UID: \"b32c919d-6838-47bd-ad43-56804a8f3937\") " pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:39:25.068202 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:25.068049    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwd2x\" (UniqueName: \"kubernetes.io/projected/b32c919d-6838-47bd-ad43-56804a8f3937-kube-api-access-rwd2x\") pod \"success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl\" (UID: \"b32c919d-6838-47bd-ad43-56804a8f3937\") " pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:39:25.068202 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:25.068100    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b32c919d-6838-47bd-ad43-56804a8f3937-proxy-tls\") pod \"success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl\" (UID: \"b32c919d-6838-47bd-ad43-56804a8f3937\") " pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:39:25.068333 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:39:25.068242    2570 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-serving-cert: secret "success-200-isvc-2bc9c-predictor-serving-cert" not found
Jul 02 13:39:25.068406 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:39:25.068334    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32c919d-6838-47bd-ad43-56804a8f3937-proxy-tls podName:b32c919d-6838-47bd-ad43-56804a8f3937 nodeName:}" failed. No retries permitted until 2026-07-02 13:39:25.568313885 +0000 UTC m=+1517.191106777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b32c919d-6838-47bd-ad43-56804a8f3937-proxy-tls") pod "success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl" (UID: "b32c919d-6838-47bd-ad43-56804a8f3937") : secret "success-200-isvc-2bc9c-predictor-serving-cert" not found
Jul 02 13:39:25.068737 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:25.068716    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b32c919d-6838-47bd-ad43-56804a8f3937-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl\" (UID: \"b32c919d-6838-47bd-ad43-56804a8f3937\") " pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:39:25.076930 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:25.076907    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwd2x\" (UniqueName: \"kubernetes.io/projected/b32c919d-6838-47bd-ad43-56804a8f3937-kube-api-access-rwd2x\") pod \"success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl\" (UID: \"b32c919d-6838-47bd-ad43-56804a8f3937\") " pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:39:25.162738 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:25.162699    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:39:25.262887 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:25.262859    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:39:25.363073 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:25.362986    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:39:25.567108 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:25.567079    2570 generic.go:356] "Generic (PLEG): container finished" podID="beb4e33f-3c05-4075-b592-f5951759c36a" containerID="6d512c29bf3b7030322992dfe1c944804d0ef164c91e11415629d7420e7bd27c" exitCode=2
Jul 02 13:39:25.567276 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:25.567149    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz" event={"ID":"beb4e33f-3c05-4075-b592-f5951759c36a","Type":"ContainerDied","Data":"6d512c29bf3b7030322992dfe1c944804d0ef164c91e11415629d7420e7bd27c"}
Jul 02 13:39:25.573509 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:25.573491    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b32c919d-6838-47bd-ad43-56804a8f3937-proxy-tls\") pod \"success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl\" (UID: \"b32c919d-6838-47bd-ad43-56804a8f3937\") " pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:39:25.575980 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:25.575958    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b32c919d-6838-47bd-ad43-56804a8f3937-proxy-tls\") pod \"success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl\" (UID: \"b32c919d-6838-47bd-ad43-56804a8f3937\") " pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:39:25.826676 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:25.826641    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:39:25.960250 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:25.959953    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"]
Jul 02 13:39:25.962977 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:39:25.962949    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb32c919d_6838_47bd_ad43_56804a8f3937.slice/crio-7dfc0c9eb2493ba3aae730430f8c0167d00b043312986e4139a84027ffd64d35 WatchSource:0}: Error finding container 7dfc0c9eb2493ba3aae730430f8c0167d00b043312986e4139a84027ffd64d35: Status 404 returned error can't find the container with id 7dfc0c9eb2493ba3aae730430f8c0167d00b043312986e4139a84027ffd64d35
Jul 02 13:39:26.573354 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:26.573321    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl" event={"ID":"b32c919d-6838-47bd-ad43-56804a8f3937","Type":"ContainerStarted","Data":"b701c6f4a7957d6ff922d98f4e151b67abc5968e4bc84a7ad7e5c7c704e319a6"}
Jul 02 13:39:26.573575 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:26.573380    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl" event={"ID":"b32c919d-6838-47bd-ad43-56804a8f3937","Type":"ContainerStarted","Data":"41bff8cb795dc7e4c50ee1f53a5d20976aef89935ae14936eea33715da00eee9"}
Jul 02 13:39:26.573575 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:26.573420    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl" event={"ID":"b32c919d-6838-47bd-ad43-56804a8f3937","Type":"ContainerStarted","Data":"7dfc0c9eb2493ba3aae730430f8c0167d00b043312986e4139a84027ffd64d35"}
Jul 02 13:39:26.573575 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:26.573442    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:39:26.592422 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:26.592349    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl" podStartSLOduration=2.592329448 podStartE2EDuration="2.592329448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:39:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:39:26.592070167 +0000 UTC m=+1518.214863074" watchObservedRunningTime="2026-07-02 13:39:26.592329448 +0000 UTC m=+1518.215122354"
Jul 02 13:39:27.578617 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:27.578590    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:39:27.580164 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:27.580136    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl" podUID="b32c919d-6838-47bd-ad43-56804a8f3937" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 02 13:39:28.157992 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.157969    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:39:28.298013 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.297973    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/beb4e33f-3c05-4075-b592-f5951759c36a-proxy-tls\") pod \"beb4e33f-3c05-4075-b592-f5951759c36a\" (UID: \"beb4e33f-3c05-4075-b592-f5951759c36a\") "
Jul 02 13:39:28.298192 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.298042    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/beb4e33f-3c05-4075-b592-f5951759c36a-kube-rbac-proxy-sar-config\") pod \"beb4e33f-3c05-4075-b592-f5951759c36a\" (UID: \"beb4e33f-3c05-4075-b592-f5951759c36a\") "
Jul 02 13:39:28.298192 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.298072    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qx2xb\" (UniqueName: \"kubernetes.io/projected/beb4e33f-3c05-4075-b592-f5951759c36a-kube-api-access-qx2xb\") pod \"beb4e33f-3c05-4075-b592-f5951759c36a\" (UID: \"beb4e33f-3c05-4075-b592-f5951759c36a\") "
Jul 02 13:39:28.298490 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.298453    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beb4e33f-3c05-4075-b592-f5951759c36a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "beb4e33f-3c05-4075-b592-f5951759c36a" (UID: "beb4e33f-3c05-4075-b592-f5951759c36a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:39:28.300177 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.300149    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/beb4e33f-3c05-4075-b592-f5951759c36a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "beb4e33f-3c05-4075-b592-f5951759c36a" (UID: "beb4e33f-3c05-4075-b592-f5951759c36a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:39:28.300177 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.300160    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/beb4e33f-3c05-4075-b592-f5951759c36a-kube-api-access-qx2xb" (OuterVolumeSpecName: "kube-api-access-qx2xb") pod "beb4e33f-3c05-4075-b592-f5951759c36a" (UID: "beb4e33f-3c05-4075-b592-f5951759c36a"). InnerVolumeSpecName "kube-api-access-qx2xb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:39:28.399210 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.399176    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/beb4e33f-3c05-4075-b592-f5951759c36a-proxy-tls\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:39:28.399210 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.399203    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/beb4e33f-3c05-4075-b592-f5951759c36a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:39:28.399210 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.399215    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx2xb\" (UniqueName: \"kubernetes.io/projected/beb4e33f-3c05-4075-b592-f5951759c36a-kube-api-access-qx2xb\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:39:28.584583 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.584493    2570 generic.go:356] "Generic (PLEG): container finished" podID="beb4e33f-3c05-4075-b592-f5951759c36a" containerID="238578c5e807d8a684dfe63c89fe4f64eb7d22a19b2b403772236151c4ec334f" exitCode=0
Jul 02 13:39:28.584583 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.584543    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"
Jul 02 13:39:28.584583 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.584576    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz" event={"ID":"beb4e33f-3c05-4075-b592-f5951759c36a","Type":"ContainerDied","Data":"238578c5e807d8a684dfe63c89fe4f64eb7d22a19b2b403772236151c4ec334f"}
Jul 02 13:39:28.585102 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.584634    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz" event={"ID":"beb4e33f-3c05-4075-b592-f5951759c36a","Type":"ContainerDied","Data":"4e78111190ba6333efeff5e3535650b2f75c33703ca9204aef21472295e3f746"}
Jul 02 13:39:28.585102 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.584663    2570 scope.go:117] "RemoveContainer" containerID="6d512c29bf3b7030322992dfe1c944804d0ef164c91e11415629d7420e7bd27c"
Jul 02 13:39:28.585299 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.585272    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl" podUID="b32c919d-6838-47bd-ad43-56804a8f3937" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 02 13:39:28.593836 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.593818    2570 scope.go:117] "RemoveContainer" containerID="238578c5e807d8a684dfe63c89fe4f64eb7d22a19b2b403772236151c4ec334f"
Jul 02 13:39:28.601263 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.601247    2570 scope.go:117] "RemoveContainer" containerID="6d512c29bf3b7030322992dfe1c944804d0ef164c91e11415629d7420e7bd27c"
Jul 02 13:39:28.602033 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:39:28.602010    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d512c29bf3b7030322992dfe1c944804d0ef164c91e11415629d7420e7bd27c\": container with ID starting with 6d512c29bf3b7030322992dfe1c944804d0ef164c91e11415629d7420e7bd27c not found: ID does not exist" containerID="6d512c29bf3b7030322992dfe1c944804d0ef164c91e11415629d7420e7bd27c"
Jul 02 13:39:28.602106 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.602040    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d512c29bf3b7030322992dfe1c944804d0ef164c91e11415629d7420e7bd27c"} err="failed to get container status \"6d512c29bf3b7030322992dfe1c944804d0ef164c91e11415629d7420e7bd27c\": rpc error: code = NotFound desc = could not find container \"6d512c29bf3b7030322992dfe1c944804d0ef164c91e11415629d7420e7bd27c\": container with ID starting with 6d512c29bf3b7030322992dfe1c944804d0ef164c91e11415629d7420e7bd27c not found: ID does not exist"
Jul 02 13:39:28.602106 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.602057    2570 scope.go:117] "RemoveContainer" containerID="238578c5e807d8a684dfe63c89fe4f64eb7d22a19b2b403772236151c4ec334f"
Jul 02 13:39:28.602327 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:39:28.602308    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"238578c5e807d8a684dfe63c89fe4f64eb7d22a19b2b403772236151c4ec334f\": container with ID starting with 238578c5e807d8a684dfe63c89fe4f64eb7d22a19b2b403772236151c4ec334f not found: ID does not exist" containerID="238578c5e807d8a684dfe63c89fe4f64eb7d22a19b2b403772236151c4ec334f"
Jul 02 13:39:28.602377 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.602332    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"238578c5e807d8a684dfe63c89fe4f64eb7d22a19b2b403772236151c4ec334f"} err="failed to get container status \"238578c5e807d8a684dfe63c89fe4f64eb7d22a19b2b403772236151c4ec334f\": rpc error: code = NotFound desc = could not find container \"238578c5e807d8a684dfe63c89fe4f64eb7d22a19b2b403772236151c4ec334f\": container with ID starting with 238578c5e807d8a684dfe63c89fe4f64eb7d22a19b2b403772236151c4ec334f not found: ID does not exist"
Jul 02 13:39:28.605481 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.605461    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"]
Jul 02 13:39:28.613422 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.610108    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f369f-predictor-7957879bc7-pvbxz"]
Jul 02 13:39:28.897761 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:28.897686    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="beb4e33f-3c05-4075-b592-f5951759c36a" path="/var/lib/kubelet/pods/beb4e33f-3c05-4075-b592-f5951759c36a/volumes"
Jul 02 13:39:32.390585 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:32.390512    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:39:33.590087 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:33.590063    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:39:33.590539 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:33.590514    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl" podUID="b32c919d-6838-47bd-ad43-56804a8f3937" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 02 13:39:43.591112 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:43.591071    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl" podUID="b32c919d-6838-47bd-ad43-56804a8f3937" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 02 13:39:45.248600 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:45.248564    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:39:45.493698 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:45.493652    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:39:53.591004 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:53.590962    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl" podUID="b32c919d-6838-47bd-ad43-56804a8f3937" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 02 13:39:55.328844 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.328802    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:39:55.411940 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.411898    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:39:55.432172 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.432141    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"]
Jul 02 13:39:55.432473 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.432451    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f" podUID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerName="kserve-container" containerID="cri-o://5112e33ac831bbaea927e5543fbe02be2181899f82225f432413fbbeede6077a" gracePeriod=30
Jul 02 13:39:55.432576 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.432509    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f" podUID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerName="kube-rbac-proxy" containerID="cri-o://8a091393383977a3f361c4e8039330c52c74a7df90c4eeebdea90efb69a92f0b" gracePeriod=30
Jul 02 13:39:55.445007 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.444978    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"]
Jul 02 13:39:55.445591 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.445569    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beb4e33f-3c05-4075-b592-f5951759c36a" containerName="kube-rbac-proxy"
Jul 02 13:39:55.445591 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.445590    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="beb4e33f-3c05-4075-b592-f5951759c36a" containerName="kube-rbac-proxy"
Jul 02 13:39:55.445791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.445631    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beb4e33f-3c05-4075-b592-f5951759c36a" containerName="kserve-container"
Jul 02 13:39:55.445791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.445640    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="beb4e33f-3c05-4075-b592-f5951759c36a" containerName="kserve-container"
Jul 02 13:39:55.445791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.445716    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="beb4e33f-3c05-4075-b592-f5951759c36a" containerName="kserve-container"
Jul 02 13:39:55.445791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.445730    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="beb4e33f-3c05-4075-b592-f5951759c36a" containerName="kube-rbac-proxy"
Jul 02 13:39:55.449395 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.449361    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:39:55.451349 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.451328    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-e2bd2-predictor-serving-cert\""
Jul 02 13:39:55.451527 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.451374    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-e2bd2-kube-rbac-proxy-sar-config\""
Jul 02 13:39:55.455630 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.455608    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"]
Jul 02 13:39:55.509792 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.509757    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:39:55.545618 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.545586    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff4c6996-9d23-4dd8-b001-ed01cf58a717-proxy-tls\") pod \"success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc\" (UID: \"ff4c6996-9d23-4dd8-b001-ed01cf58a717\") " pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:39:55.545782 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.545714    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ff4c6996-9d23-4dd8-b001-ed01cf58a717-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc\" (UID: \"ff4c6996-9d23-4dd8-b001-ed01cf58a717\") " pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:39:55.545782 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.545772    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvhrq\" (UniqueName: \"kubernetes.io/projected/ff4c6996-9d23-4dd8-b001-ed01cf58a717-kube-api-access-dvhrq\") pod \"success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc\" (UID: \"ff4c6996-9d23-4dd8-b001-ed01cf58a717\") " pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:39:55.575503 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.575467    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:39:55.647070 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.646979    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff4c6996-9d23-4dd8-b001-ed01cf58a717-proxy-tls\") pod \"success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc\" (UID: \"ff4c6996-9d23-4dd8-b001-ed01cf58a717\") " pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:39:55.647252 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:39:55.647106    2570 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-serving-cert: secret "success-200-isvc-e2bd2-predictor-serving-cert" not found
Jul 02 13:39:55.647252 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.647137    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ff4c6996-9d23-4dd8-b001-ed01cf58a717-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc\" (UID: \"ff4c6996-9d23-4dd8-b001-ed01cf58a717\") " pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:39:55.647252 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:39:55.647186    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4c6996-9d23-4dd8-b001-ed01cf58a717-proxy-tls podName:ff4c6996-9d23-4dd8-b001-ed01cf58a717 nodeName:}" failed. No retries permitted until 2026-07-02 13:39:56.14716719 +0000 UTC m=+1547.769960074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/ff4c6996-9d23-4dd8-b001-ed01cf58a717-proxy-tls") pod "success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc" (UID: "ff4c6996-9d23-4dd8-b001-ed01cf58a717") : secret "success-200-isvc-e2bd2-predictor-serving-cert" not found
Jul 02 13:39:55.647252 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.647218    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvhrq\" (UniqueName: \"kubernetes.io/projected/ff4c6996-9d23-4dd8-b001-ed01cf58a717-kube-api-access-dvhrq\") pod \"success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc\" (UID: \"ff4c6996-9d23-4dd8-b001-ed01cf58a717\") " pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:39:55.647854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.647829    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ff4c6996-9d23-4dd8-b001-ed01cf58a717-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc\" (UID: \"ff4c6996-9d23-4dd8-b001-ed01cf58a717\") " pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:39:55.654848 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.654819    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvhrq\" (UniqueName: \"kubernetes.io/projected/ff4c6996-9d23-4dd8-b001-ed01cf58a717-kube-api-access-dvhrq\") pod \"success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc\" (UID: \"ff4c6996-9d23-4dd8-b001-ed01cf58a717\") " pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:39:55.713536 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.713501    2570 generic.go:356] "Generic (PLEG): container finished" podID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerID="8a091393383977a3f361c4e8039330c52c74a7df90c4eeebdea90efb69a92f0b" exitCode=2
Jul 02 13:39:55.713688 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.713557    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f" event={"ID":"328ec472-00f3-45f9-ab8b-9ae649b29ee8","Type":"ContainerDied","Data":"8a091393383977a3f361c4e8039330c52c74a7df90c4eeebdea90efb69a92f0b"}
Jul 02 13:39:55.763398 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:55.763344    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:39:56.097105 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:56.097064    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:39:56.150535 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:56.150504    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff4c6996-9d23-4dd8-b001-ed01cf58a717-proxy-tls\") pod \"success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc\" (UID: \"ff4c6996-9d23-4dd8-b001-ed01cf58a717\") " pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:39:56.152788 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:56.152766    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff4c6996-9d23-4dd8-b001-ed01cf58a717-proxy-tls\") pod \"success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc\" (UID: \"ff4c6996-9d23-4dd8-b001-ed01cf58a717\") " pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:39:56.362206 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:56.362111    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:39:56.492933 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:56.492907    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"]
Jul 02 13:39:56.494535 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:39:56.494511    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff4c6996_9d23_4dd8_b001_ed01cf58a717.slice/crio-fd6be8f786ad9765cf96021bae0f8117de9c2fbec8fad2057930d2c40710f7a1 WatchSource:0}: Error finding container fd6be8f786ad9765cf96021bae0f8117de9c2fbec8fad2057930d2c40710f7a1: Status 404 returned error can't find the container with id fd6be8f786ad9765cf96021bae0f8117de9c2fbec8fad2057930d2c40710f7a1
Jul 02 13:39:56.720433 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:56.720380    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc" event={"ID":"ff4c6996-9d23-4dd8-b001-ed01cf58a717","Type":"ContainerStarted","Data":"67682ff14e7d7c3a8dc78951013094bbbfc0faf9d04aaa5bdc3c999b72e7d0c7"}
Jul 02 13:39:56.720632 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:56.720459    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc" event={"ID":"ff4c6996-9d23-4dd8-b001-ed01cf58a717","Type":"ContainerStarted","Data":"0b4779504c7f195327c40dbccbadf373bbf36d2b85bb54ad92ea90834755a92e"}
Jul 02 13:39:56.720632 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:56.720488    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc" event={"ID":"ff4c6996-9d23-4dd8-b001-ed01cf58a717","Type":"ContainerStarted","Data":"fd6be8f786ad9765cf96021bae0f8117de9c2fbec8fad2057930d2c40710f7a1"}
Jul 02 13:39:56.720754 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:56.720700    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:39:56.737512 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:56.737440    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc" podStartSLOduration=1.7374178630000001 podStartE2EDuration="1.737417863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:39:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:39:56.734620842 +0000 UTC m=+1548.357413749" watchObservedRunningTime="2026-07-02 13:39:56.737417863 +0000 UTC m=+1548.360210773"
Jul 02 13:39:57.384793 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:57.384752    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f" podUID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.37:8643/healthz\": dial tcp 10.132.0.37:8643: connect: connection refused"
Jul 02 13:39:57.725848 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:57.725819    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:39:57.727273 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:57.727244    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc" podUID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 02 13:39:58.733157 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:58.733125    2570 generic.go:356] "Generic (PLEG): container finished" podID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerID="5112e33ac831bbaea927e5543fbe02be2181899f82225f432413fbbeede6077a" exitCode=0
Jul 02 13:39:58.733632 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:58.733190    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f" event={"ID":"328ec472-00f3-45f9-ab8b-9ae649b29ee8","Type":"ContainerDied","Data":"5112e33ac831bbaea927e5543fbe02be2181899f82225f432413fbbeede6077a"}
Jul 02 13:39:58.733632 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:58.733500    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc" podUID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 02 13:39:58.790662 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:58.790604    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:39:58.874766 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:58.874731    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/328ec472-00f3-45f9-ab8b-9ae649b29ee8-kube-rbac-proxy-sar-config\") pod \"328ec472-00f3-45f9-ab8b-9ae649b29ee8\" (UID: \"328ec472-00f3-45f9-ab8b-9ae649b29ee8\") "
Jul 02 13:39:58.874944 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:58.874789    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/328ec472-00f3-45f9-ab8b-9ae649b29ee8-proxy-tls\") pod \"328ec472-00f3-45f9-ab8b-9ae649b29ee8\" (UID: \"328ec472-00f3-45f9-ab8b-9ae649b29ee8\") "
Jul 02 13:39:58.874944 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:58.874815    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tk7wt\" (UniqueName: \"kubernetes.io/projected/328ec472-00f3-45f9-ab8b-9ae649b29ee8-kube-api-access-tk7wt\") pod \"328ec472-00f3-45f9-ab8b-9ae649b29ee8\" (UID: \"328ec472-00f3-45f9-ab8b-9ae649b29ee8\") "
Jul 02 13:39:58.875119 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:58.875095    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/328ec472-00f3-45f9-ab8b-9ae649b29ee8-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "328ec472-00f3-45f9-ab8b-9ae649b29ee8" (UID: "328ec472-00f3-45f9-ab8b-9ae649b29ee8"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:39:58.876944 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:58.876909    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/328ec472-00f3-45f9-ab8b-9ae649b29ee8-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "328ec472-00f3-45f9-ab8b-9ae649b29ee8" (UID: "328ec472-00f3-45f9-ab8b-9ae649b29ee8"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:39:58.876944 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:58.876923    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/328ec472-00f3-45f9-ab8b-9ae649b29ee8-kube-api-access-tk7wt" (OuterVolumeSpecName: "kube-api-access-tk7wt") pod "328ec472-00f3-45f9-ab8b-9ae649b29ee8" (UID: "328ec472-00f3-45f9-ab8b-9ae649b29ee8"). InnerVolumeSpecName "kube-api-access-tk7wt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:39:58.975701 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:58.975663    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/328ec472-00f3-45f9-ab8b-9ae649b29ee8-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:39:58.975701 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:58.975694    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/328ec472-00f3-45f9-ab8b-9ae649b29ee8-proxy-tls\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:39:58.975701 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:58.975705    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tk7wt\" (UniqueName: \"kubernetes.io/projected/328ec472-00f3-45f9-ab8b-9ae649b29ee8-kube-api-access-tk7wt\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:39:59.740676 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:59.740640    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f" event={"ID":"328ec472-00f3-45f9-ab8b-9ae649b29ee8","Type":"ContainerDied","Data":"688e35cbd65297de40a8e812cd0fefa20404cf4665166c973807c2c04b13a5b3"}
Jul 02 13:39:59.741070 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:59.740711    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"
Jul 02 13:39:59.741070 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:59.740737    2570 scope.go:117] "RemoveContainer" containerID="8a091393383977a3f361c4e8039330c52c74a7df90c4eeebdea90efb69a92f0b"
Jul 02 13:39:59.749368 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:59.749348    2570 scope.go:117] "RemoveContainer" containerID="5112e33ac831bbaea927e5543fbe02be2181899f82225f432413fbbeede6077a"
Jul 02 13:39:59.755224 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:59.755197    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"]
Jul 02 13:39:59.758436 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:39:59.758416    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6c5e8-predictor-866c9df647-kc54f"]
Jul 02 13:40:00.898375 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:00.898341    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" path="/var/lib/kubelet/pods/328ec472-00f3-45f9-ab8b-9ae649b29ee8/volumes"
Jul 02 13:40:03.591002 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:03.590957    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl" podUID="b32c919d-6838-47bd-ad43-56804a8f3937" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 02 13:40:03.739004 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:03.738969    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:40:03.739545 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:03.739517    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc" podUID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 02 13:40:13.591153 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:13.591120    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:40:13.739627 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:13.739584    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc" podUID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 02 13:40:23.739638 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:23.739589    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc" podUID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 02 13:40:24.927003 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:24.926955    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:40:25.082793 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:25.082762    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:40:33.740299 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:33.740247    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc" podUID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 02 13:40:35.020150 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.020114    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:40:35.101669 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.101622    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:40:35.123317 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.123287    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"]
Jul 02 13:40:35.123874 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.123852    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerName="kserve-container"
Jul 02 13:40:35.123874 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.123873    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerName="kserve-container"
Jul 02 13:40:35.124072 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.123888    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerName="kube-rbac-proxy"
Jul 02 13:40:35.124072 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.123896    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerName="kube-rbac-proxy"
Jul 02 13:40:35.124072 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.124013    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerName="kube-rbac-proxy"
Jul 02 13:40:35.124072 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.124026    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="328ec472-00f3-45f9-ab8b-9ae649b29ee8" containerName="kserve-container"
Jul 02 13:40:35.127426 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.127404    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:40:35.128954 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.128929    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"]
Jul 02 13:40:35.129164 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.129137    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl" podUID="b32c919d-6838-47bd-ad43-56804a8f3937" containerName="kserve-container" containerID="cri-o://41bff8cb795dc7e4c50ee1f53a5d20976aef89935ae14936eea33715da00eee9" gracePeriod=30
Jul 02 13:40:35.129280 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.129137    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl" podUID="b32c919d-6838-47bd-ad43-56804a8f3937" containerName="kube-rbac-proxy" containerID="cri-o://b701c6f4a7957d6ff922d98f4e151b67abc5968e4bc84a7ad7e5c7c704e319a6" gracePeriod=30
Jul 02 13:40:35.129954 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.129552    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-7b15c-predictor-serving-cert\""
Jul 02 13:40:35.129954 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.129759    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-7b15c-kube-rbac-proxy-sar-config\""
Jul 02 13:40:35.135900 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.135873    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"]
Jul 02 13:40:35.168813 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.168776    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:40:35.233883 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.233843    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:40:35.286649 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.286553    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f1a08288-183d-4a3d-8202-cd0b559e5a05-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc\" (UID: \"f1a08288-183d-4a3d-8202-cd0b559e5a05\") " pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:40:35.286881 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.286860    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6mmh\" (UniqueName: \"kubernetes.io/projected/f1a08288-183d-4a3d-8202-cd0b559e5a05-kube-api-access-f6mmh\") pod \"success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc\" (UID: \"f1a08288-183d-4a3d-8202-cd0b559e5a05\") " pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:40:35.287106 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.287086    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f1a08288-183d-4a3d-8202-cd0b559e5a05-proxy-tls\") pod \"success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc\" (UID: \"f1a08288-183d-4a3d-8202-cd0b559e5a05\") " pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:40:35.365314 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.365276    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:40:35.387988 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.387931    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f1a08288-183d-4a3d-8202-cd0b559e5a05-proxy-tls\") pod \"success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc\" (UID: \"f1a08288-183d-4a3d-8202-cd0b559e5a05\") " pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:40:35.388147 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.388037    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f1a08288-183d-4a3d-8202-cd0b559e5a05-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc\" (UID: \"f1a08288-183d-4a3d-8202-cd0b559e5a05\") " pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:40:35.388147 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.388070    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6mmh\" (UniqueName: \"kubernetes.io/projected/f1a08288-183d-4a3d-8202-cd0b559e5a05-kube-api-access-f6mmh\") pod \"success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc\" (UID: \"f1a08288-183d-4a3d-8202-cd0b559e5a05\") " pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:40:35.388779 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.388754    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f1a08288-183d-4a3d-8202-cd0b559e5a05-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc\" (UID: \"f1a08288-183d-4a3d-8202-cd0b559e5a05\") " pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:40:35.390454 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.390434    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f1a08288-183d-4a3d-8202-cd0b559e5a05-proxy-tls\") pod \"success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc\" (UID: \"f1a08288-183d-4a3d-8202-cd0b559e5a05\") " pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:40:35.395112 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.395088    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6mmh\" (UniqueName: \"kubernetes.io/projected/f1a08288-183d-4a3d-8202-cd0b559e5a05-kube-api-access-f6mmh\") pod \"success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc\" (UID: \"f1a08288-183d-4a3d-8202-cd0b559e5a05\") " pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:40:35.451039 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.451011    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:40:35.580155 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.580131    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"]
Jul 02 13:40:35.582541 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:40:35.582517    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1a08288_183d_4a3d_8202_cd0b559e5a05.slice/crio-ce948ff0b43496b82d54a69feedfb2e25b5e24d5ef939d009e71fdcea591ece6 WatchSource:0}: Error finding container ce948ff0b43496b82d54a69feedfb2e25b5e24d5ef939d009e71fdcea591ece6: Status 404 returned error can't find the container with id ce948ff0b43496b82d54a69feedfb2e25b5e24d5ef939d009e71fdcea591ece6
Jul 02 13:40:35.919576 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.919493    2570 generic.go:356] "Generic (PLEG): container finished" podID="b32c919d-6838-47bd-ad43-56804a8f3937" containerID="b701c6f4a7957d6ff922d98f4e151b67abc5968e4bc84a7ad7e5c7c704e319a6" exitCode=2
Jul 02 13:40:35.919729 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.919570    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl" event={"ID":"b32c919d-6838-47bd-ad43-56804a8f3937","Type":"ContainerDied","Data":"b701c6f4a7957d6ff922d98f4e151b67abc5968e4bc84a7ad7e5c7c704e319a6"}
Jul 02 13:40:35.921178 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.921154    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" event={"ID":"f1a08288-183d-4a3d-8202-cd0b559e5a05","Type":"ContainerStarted","Data":"a02ea7e7a37b0ff1e1c1dd77a811974948c80759819215719e0b3aff965c3057"}
Jul 02 13:40:35.921296 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.921199    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" event={"ID":"f1a08288-183d-4a3d-8202-cd0b559e5a05","Type":"ContainerStarted","Data":"63ce6df5448a7c558b55c2195d6df4c3fc25cafb5c27212c45f3a6af92939e1e"}
Jul 02 13:40:35.921296 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.921222    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" event={"ID":"f1a08288-183d-4a3d-8202-cd0b559e5a05","Type":"ContainerStarted","Data":"ce948ff0b43496b82d54a69feedfb2e25b5e24d5ef939d009e71fdcea591ece6"}
Jul 02 13:40:35.921296 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.921273    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:40:35.936457 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:35.936367    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" podStartSLOduration=0.936346556 podStartE2EDuration="936.346556ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:40:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:40:35.93497694 +0000 UTC m=+1587.557769851" watchObservedRunningTime="2026-07-02 13:40:35.936346556 +0000 UTC m=+1587.559139467"
Jul 02 13:40:36.925632 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:36.925605    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:40:36.927136 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:36.927106    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 02 13:40:37.930439 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:37.930374    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 02 13:40:38.582438 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.582412    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:40:38.617321 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.617222    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwd2x\" (UniqueName: \"kubernetes.io/projected/b32c919d-6838-47bd-ad43-56804a8f3937-kube-api-access-rwd2x\") pod \"b32c919d-6838-47bd-ad43-56804a8f3937\" (UID: \"b32c919d-6838-47bd-ad43-56804a8f3937\") "
Jul 02 13:40:38.617321 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.617261    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b32c919d-6838-47bd-ad43-56804a8f3937-proxy-tls\") pod \"b32c919d-6838-47bd-ad43-56804a8f3937\" (UID: \"b32c919d-6838-47bd-ad43-56804a8f3937\") "
Jul 02 13:40:38.617321 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.617299    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b32c919d-6838-47bd-ad43-56804a8f3937-kube-rbac-proxy-sar-config\") pod \"b32c919d-6838-47bd-ad43-56804a8f3937\" (UID: \"b32c919d-6838-47bd-ad43-56804a8f3937\") "
Jul 02 13:40:38.617718 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.617695    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b32c919d-6838-47bd-ad43-56804a8f3937-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b32c919d-6838-47bd-ad43-56804a8f3937" (UID: "b32c919d-6838-47bd-ad43-56804a8f3937"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:40:38.619352 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.619327    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b32c919d-6838-47bd-ad43-56804a8f3937-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b32c919d-6838-47bd-ad43-56804a8f3937" (UID: "b32c919d-6838-47bd-ad43-56804a8f3937"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:40:38.619491 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.619443    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b32c919d-6838-47bd-ad43-56804a8f3937-kube-api-access-rwd2x" (OuterVolumeSpecName: "kube-api-access-rwd2x") pod "b32c919d-6838-47bd-ad43-56804a8f3937" (UID: "b32c919d-6838-47bd-ad43-56804a8f3937"). InnerVolumeSpecName "kube-api-access-rwd2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:40:38.718058 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.717998    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwd2x\" (UniqueName: \"kubernetes.io/projected/b32c919d-6838-47bd-ad43-56804a8f3937-kube-api-access-rwd2x\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:40:38.718058 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.718050    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b32c919d-6838-47bd-ad43-56804a8f3937-proxy-tls\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:40:38.718058 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.718060    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b32c919d-6838-47bd-ad43-56804a8f3937-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:40:38.936952 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.936914    2570 generic.go:356] "Generic (PLEG): container finished" podID="b32c919d-6838-47bd-ad43-56804a8f3937" containerID="41bff8cb795dc7e4c50ee1f53a5d20976aef89935ae14936eea33715da00eee9" exitCode=0
Jul 02 13:40:38.937355 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.936961    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"
Jul 02 13:40:38.937355 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.936961    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl" event={"ID":"b32c919d-6838-47bd-ad43-56804a8f3937","Type":"ContainerDied","Data":"41bff8cb795dc7e4c50ee1f53a5d20976aef89935ae14936eea33715da00eee9"}
Jul 02 13:40:38.937355 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.937029    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl" event={"ID":"b32c919d-6838-47bd-ad43-56804a8f3937","Type":"ContainerDied","Data":"7dfc0c9eb2493ba3aae730430f8c0167d00b043312986e4139a84027ffd64d35"}
Jul 02 13:40:38.937355 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.937074    2570 scope.go:117] "RemoveContainer" containerID="b701c6f4a7957d6ff922d98f4e151b67abc5968e4bc84a7ad7e5c7c704e319a6"
Jul 02 13:40:38.946243 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.946217    2570 scope.go:117] "RemoveContainer" containerID="41bff8cb795dc7e4c50ee1f53a5d20976aef89935ae14936eea33715da00eee9"
Jul 02 13:40:38.952046 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.952019    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"]
Jul 02 13:40:38.955208 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.955111    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2bc9c-predictor-59ff898c87-zfgzl"]
Jul 02 13:40:38.955264 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.955234    2570 scope.go:117] "RemoveContainer" containerID="b701c6f4a7957d6ff922d98f4e151b67abc5968e4bc84a7ad7e5c7c704e319a6"
Jul 02 13:40:38.955529 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:40:38.955508    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b701c6f4a7957d6ff922d98f4e151b67abc5968e4bc84a7ad7e5c7c704e319a6\": container with ID starting with b701c6f4a7957d6ff922d98f4e151b67abc5968e4bc84a7ad7e5c7c704e319a6 not found: ID does not exist" containerID="b701c6f4a7957d6ff922d98f4e151b67abc5968e4bc84a7ad7e5c7c704e319a6"
Jul 02 13:40:38.955570 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.955539    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b701c6f4a7957d6ff922d98f4e151b67abc5968e4bc84a7ad7e5c7c704e319a6"} err="failed to get container status \"b701c6f4a7957d6ff922d98f4e151b67abc5968e4bc84a7ad7e5c7c704e319a6\": rpc error: code = NotFound desc = could not find container \"b701c6f4a7957d6ff922d98f4e151b67abc5968e4bc84a7ad7e5c7c704e319a6\": container with ID starting with b701c6f4a7957d6ff922d98f4e151b67abc5968e4bc84a7ad7e5c7c704e319a6 not found: ID does not exist"
Jul 02 13:40:38.955570 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.955558    2570 scope.go:117] "RemoveContainer" containerID="41bff8cb795dc7e4c50ee1f53a5d20976aef89935ae14936eea33715da00eee9"
Jul 02 13:40:38.955779 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:40:38.955765    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41bff8cb795dc7e4c50ee1f53a5d20976aef89935ae14936eea33715da00eee9\": container with ID starting with 41bff8cb795dc7e4c50ee1f53a5d20976aef89935ae14936eea33715da00eee9 not found: ID does not exist" containerID="41bff8cb795dc7e4c50ee1f53a5d20976aef89935ae14936eea33715da00eee9"
Jul 02 13:40:38.955815 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:38.955783    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41bff8cb795dc7e4c50ee1f53a5d20976aef89935ae14936eea33715da00eee9"} err="failed to get container status \"41bff8cb795dc7e4c50ee1f53a5d20976aef89935ae14936eea33715da00eee9\": rpc error: code = NotFound desc = could not find container \"41bff8cb795dc7e4c50ee1f53a5d20976aef89935ae14936eea33715da00eee9\": container with ID starting with 41bff8cb795dc7e4c50ee1f53a5d20976aef89935ae14936eea33715da00eee9 not found: ID does not exist"
Jul 02 13:40:40.897898 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:40.897863    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b32c919d-6838-47bd-ad43-56804a8f3937" path="/var/lib/kubelet/pods/b32c919d-6838-47bd-ad43-56804a8f3937/volumes"
Jul 02 13:40:42.934541 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:42.934506    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:40:42.935071 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:42.935041    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 02 13:40:43.740582 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:43.740549    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:40:52.936055 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:52.936018    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 02 13:40:55.513012 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:55.512983    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:40:56.938920 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:40:56.938876    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:41:02.935222 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:41:02.935137    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 02 13:41:12.935320 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:41:12.935275    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 02 13:41:22.936118 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:41:22.936090    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:41:27.962605 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:41:27.962565    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:41:35.191414 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:41:35.191360    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:41:35.376048 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:41:35.376001    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:43:24.722184 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:43:24.722144    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:44:00.219274 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:44:00.219237    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:44:08.984003 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:44:08.983977    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:44:08.987705 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:44:08.987682    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:44:08.989905 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:44:08.989882    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:44:08.993621 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:44:08.993602    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:48:13.936273 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:48:13.936245    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:39.928072 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:48:39.927991    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:48:40.028871 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:48:40.028839    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:09.016095 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:09.016066    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:49:09.020083 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:09.020058    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:49:09.022489 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:09.022470    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:49:09.026873 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:09.026847    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:49:15.262913 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.262877    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:15.377419 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.377368    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:15.415424 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.415373    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"]
Jul 02 13:49:15.415911 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.415888    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b32c919d-6838-47bd-ad43-56804a8f3937" containerName="kserve-container"
Jul 02 13:49:15.415911 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.415910    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b32c919d-6838-47bd-ad43-56804a8f3937" containerName="kserve-container"
Jul 02 13:49:15.416156 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.415939    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b32c919d-6838-47bd-ad43-56804a8f3937" containerName="kube-rbac-proxy"
Jul 02 13:49:15.416156 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.415947    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b32c919d-6838-47bd-ad43-56804a8f3937" containerName="kube-rbac-proxy"
Jul 02 13:49:15.416156 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.416051    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b32c919d-6838-47bd-ad43-56804a8f3937" containerName="kserve-container"
Jul 02 13:49:15.416156 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.416062    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b32c919d-6838-47bd-ad43-56804a8f3937" containerName="kube-rbac-proxy"
Jul 02 13:49:15.420535 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.420513    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:49:15.422332 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.422295    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-9633b-predictor-serving-cert\""
Jul 02 13:49:15.422529 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.422298    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-9633b-kube-rbac-proxy-sar-config\""
Jul 02 13:49:15.436800 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.436650    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"]
Jul 02 13:49:15.441685 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.440750    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"]
Jul 02 13:49:15.441685 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.440999    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc" podUID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" containerName="kserve-container" containerID="cri-o://0b4779504c7f195327c40dbccbadf373bbf36d2b85bb54ad92ea90834755a92e" gracePeriod=30
Jul 02 13:49:15.441685 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.441338    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc" podUID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" containerName="kube-rbac-proxy" containerID="cri-o://67682ff14e7d7c3a8dc78951013094bbbfc0faf9d04aaa5bdc3c999b72e7d0c7" gracePeriod=30
Jul 02 13:49:15.468904 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.468861    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:15.528428 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.528313    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:15.557335 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.557298    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6cdg\" (UniqueName: \"kubernetes.io/projected/27261a17-2625-4494-bd67-c89c19d41f32-kube-api-access-n6cdg\") pod \"success-200-isvc-9633b-predictor-774c89fdf5-kzkcr\" (UID: \"27261a17-2625-4494-bd67-c89c19d41f32\") " pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:49:15.557508 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.557349    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/27261a17-2625-4494-bd67-c89c19d41f32-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-9633b-predictor-774c89fdf5-kzkcr\" (UID: \"27261a17-2625-4494-bd67-c89c19d41f32\") " pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:49:15.557508 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.557442    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/27261a17-2625-4494-bd67-c89c19d41f32-proxy-tls\") pod \"success-200-isvc-9633b-predictor-774c89fdf5-kzkcr\" (UID: \"27261a17-2625-4494-bd67-c89c19d41f32\") " pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:49:15.592883 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.592849    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:15.658746 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.658707    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/27261a17-2625-4494-bd67-c89c19d41f32-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-9633b-predictor-774c89fdf5-kzkcr\" (UID: \"27261a17-2625-4494-bd67-c89c19d41f32\") " pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:49:15.658906 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.658761    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/27261a17-2625-4494-bd67-c89c19d41f32-proxy-tls\") pod \"success-200-isvc-9633b-predictor-774c89fdf5-kzkcr\" (UID: \"27261a17-2625-4494-bd67-c89c19d41f32\") " pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:49:15.658906 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.658833    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6cdg\" (UniqueName: \"kubernetes.io/projected/27261a17-2625-4494-bd67-c89c19d41f32-kube-api-access-n6cdg\") pod \"success-200-isvc-9633b-predictor-774c89fdf5-kzkcr\" (UID: \"27261a17-2625-4494-bd67-c89c19d41f32\") " pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:49:15.659368 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.659338    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/27261a17-2625-4494-bd67-c89c19d41f32-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-9633b-predictor-774c89fdf5-kzkcr\" (UID: \"27261a17-2625-4494-bd67-c89c19d41f32\") " pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:49:15.661207 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.661186    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/27261a17-2625-4494-bd67-c89c19d41f32-proxy-tls\") pod \"success-200-isvc-9633b-predictor-774c89fdf5-kzkcr\" (UID: \"27261a17-2625-4494-bd67-c89c19d41f32\") " pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:49:15.665339 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.665318    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6cdg\" (UniqueName: \"kubernetes.io/projected/27261a17-2625-4494-bd67-c89c19d41f32-kube-api-access-n6cdg\") pod \"success-200-isvc-9633b-predictor-774c89fdf5-kzkcr\" (UID: \"27261a17-2625-4494-bd67-c89c19d41f32\") " pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:49:15.740923 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.740877    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:49:15.876285 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:15.876244    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"]
Jul 02 13:49:15.878680 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:49:15.878653    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27261a17_2625_4494_bd67_c89c19d41f32.slice/crio-1cd803e9c93e47db792fac32ed102d39100bf52ca497bd4d392628682bacf405 WatchSource:0}: Error finding container 1cd803e9c93e47db792fac32ed102d39100bf52ca497bd4d392628682bacf405: Status 404 returned error can't find the container with id 1cd803e9c93e47db792fac32ed102d39100bf52ca497bd4d392628682bacf405
Jul 02 13:49:16.383609 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:16.383572    2570 generic.go:356] "Generic (PLEG): container finished" podID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" containerID="67682ff14e7d7c3a8dc78951013094bbbfc0faf9d04aaa5bdc3c999b72e7d0c7" exitCode=2
Jul 02 13:49:16.384007 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:16.383646    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc" event={"ID":"ff4c6996-9d23-4dd8-b001-ed01cf58a717","Type":"ContainerDied","Data":"67682ff14e7d7c3a8dc78951013094bbbfc0faf9d04aaa5bdc3c999b72e7d0c7"}
Jul 02 13:49:16.385399 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:16.385360    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr" event={"ID":"27261a17-2625-4494-bd67-c89c19d41f32","Type":"ContainerStarted","Data":"7ee5c2fd29ccf4a978a072d8533bd441a86163ea55d5c07cd0f19f71611e72b8"}
Jul 02 13:49:16.385521 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:16.385427    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr" event={"ID":"27261a17-2625-4494-bd67-c89c19d41f32","Type":"ContainerStarted","Data":"3c34eab6fc551ec14d94c2e69d5e7476186fbd007189e2a2611aa3ae44130761"}
Jul 02 13:49:16.385521 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:16.385453    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr" event={"ID":"27261a17-2625-4494-bd67-c89c19d41f32","Type":"ContainerStarted","Data":"1cd803e9c93e47db792fac32ed102d39100bf52ca497bd4d392628682bacf405"}
Jul 02 13:49:16.385521 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:16.385501    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:49:16.402187 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:16.402148    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr" podStartSLOduration=1.40213623 podStartE2EDuration="1.40213623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:49:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:49:16.39963563 +0000 UTC m=+2108.022428536" watchObservedRunningTime="2026-07-02 13:49:16.40213623 +0000 UTC m=+2108.024929136"
Jul 02 13:49:17.393741 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:17.393709    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:49:17.395055 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:17.395026    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr" podUID="27261a17-2625-4494-bd67-c89c19d41f32" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 02 13:49:18.398764 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:18.398723    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr" podUID="27261a17-2625-4494-bd67-c89c19d41f32" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 02 13:49:18.599694 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:18.599667    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:49:18.683767 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:18.683676    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff4c6996-9d23-4dd8-b001-ed01cf58a717-proxy-tls\") pod \"ff4c6996-9d23-4dd8-b001-ed01cf58a717\" (UID: \"ff4c6996-9d23-4dd8-b001-ed01cf58a717\") "
Jul 02 13:49:18.683767 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:18.683722    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ff4c6996-9d23-4dd8-b001-ed01cf58a717-kube-rbac-proxy-sar-config\") pod \"ff4c6996-9d23-4dd8-b001-ed01cf58a717\" (UID: \"ff4c6996-9d23-4dd8-b001-ed01cf58a717\") "
Jul 02 13:49:18.683999 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:18.683809    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvhrq\" (UniqueName: \"kubernetes.io/projected/ff4c6996-9d23-4dd8-b001-ed01cf58a717-kube-api-access-dvhrq\") pod \"ff4c6996-9d23-4dd8-b001-ed01cf58a717\" (UID: \"ff4c6996-9d23-4dd8-b001-ed01cf58a717\") "
Jul 02 13:49:18.684113 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:18.684087    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff4c6996-9d23-4dd8-b001-ed01cf58a717-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ff4c6996-9d23-4dd8-b001-ed01cf58a717" (UID: "ff4c6996-9d23-4dd8-b001-ed01cf58a717"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:49:18.685827 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:18.685805    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff4c6996-9d23-4dd8-b001-ed01cf58a717-kube-api-access-dvhrq" (OuterVolumeSpecName: "kube-api-access-dvhrq") pod "ff4c6996-9d23-4dd8-b001-ed01cf58a717" (UID: "ff4c6996-9d23-4dd8-b001-ed01cf58a717"). InnerVolumeSpecName "kube-api-access-dvhrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:49:18.685910 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:18.685882    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff4c6996-9d23-4dd8-b001-ed01cf58a717-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ff4c6996-9d23-4dd8-b001-ed01cf58a717" (UID: "ff4c6996-9d23-4dd8-b001-ed01cf58a717"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:49:18.784937 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:18.784905    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff4c6996-9d23-4dd8-b001-ed01cf58a717-proxy-tls\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:49:18.784937 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:18.784933    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ff4c6996-9d23-4dd8-b001-ed01cf58a717-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:49:18.784937 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:18.784944    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvhrq\" (UniqueName: \"kubernetes.io/projected/ff4c6996-9d23-4dd8-b001-ed01cf58a717-kube-api-access-dvhrq\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:49:19.404787 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:19.404752    2570 generic.go:356] "Generic (PLEG): container finished" podID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" containerID="0b4779504c7f195327c40dbccbadf373bbf36d2b85bb54ad92ea90834755a92e" exitCode=0
Jul 02 13:49:19.405198 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:19.404797    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc" event={"ID":"ff4c6996-9d23-4dd8-b001-ed01cf58a717","Type":"ContainerDied","Data":"0b4779504c7f195327c40dbccbadf373bbf36d2b85bb54ad92ea90834755a92e"}
Jul 02 13:49:19.405198 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:19.404805    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"
Jul 02 13:49:19.405198 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:19.404855    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc" event={"ID":"ff4c6996-9d23-4dd8-b001-ed01cf58a717","Type":"ContainerDied","Data":"fd6be8f786ad9765cf96021bae0f8117de9c2fbec8fad2057930d2c40710f7a1"}
Jul 02 13:49:19.405198 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:19.404890    2570 scope.go:117] "RemoveContainer" containerID="67682ff14e7d7c3a8dc78951013094bbbfc0faf9d04aaa5bdc3c999b72e7d0c7"
Jul 02 13:49:19.413344 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:19.413324    2570 scope.go:117] "RemoveContainer" containerID="0b4779504c7f195327c40dbccbadf373bbf36d2b85bb54ad92ea90834755a92e"
Jul 02 13:49:19.419891 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:19.419854    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"]
Jul 02 13:49:19.421492 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:19.421473    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e2bd2-predictor-6bcb66fb47-n57qc"]
Jul 02 13:49:19.421923 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:19.421867    2570 scope.go:117] "RemoveContainer" containerID="67682ff14e7d7c3a8dc78951013094bbbfc0faf9d04aaa5bdc3c999b72e7d0c7"
Jul 02 13:49:19.422165 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:49:19.422147    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67682ff14e7d7c3a8dc78951013094bbbfc0faf9d04aaa5bdc3c999b72e7d0c7\": container with ID starting with 67682ff14e7d7c3a8dc78951013094bbbfc0faf9d04aaa5bdc3c999b72e7d0c7 not found: ID does not exist" containerID="67682ff14e7d7c3a8dc78951013094bbbfc0faf9d04aaa5bdc3c999b72e7d0c7"
Jul 02 13:49:19.422211 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:19.422175    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67682ff14e7d7c3a8dc78951013094bbbfc0faf9d04aaa5bdc3c999b72e7d0c7"} err="failed to get container status \"67682ff14e7d7c3a8dc78951013094bbbfc0faf9d04aaa5bdc3c999b72e7d0c7\": rpc error: code = NotFound desc = could not find container \"67682ff14e7d7c3a8dc78951013094bbbfc0faf9d04aaa5bdc3c999b72e7d0c7\": container with ID starting with 67682ff14e7d7c3a8dc78951013094bbbfc0faf9d04aaa5bdc3c999b72e7d0c7 not found: ID does not exist"
Jul 02 13:49:19.422211 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:19.422193    2570 scope.go:117] "RemoveContainer" containerID="0b4779504c7f195327c40dbccbadf373bbf36d2b85bb54ad92ea90834755a92e"
Jul 02 13:49:19.422462 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:49:19.422445    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b4779504c7f195327c40dbccbadf373bbf36d2b85bb54ad92ea90834755a92e\": container with ID starting with 0b4779504c7f195327c40dbccbadf373bbf36d2b85bb54ad92ea90834755a92e not found: ID does not exist" containerID="0b4779504c7f195327c40dbccbadf373bbf36d2b85bb54ad92ea90834755a92e"
Jul 02 13:49:19.422507 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:19.422469    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b4779504c7f195327c40dbccbadf373bbf36d2b85bb54ad92ea90834755a92e"} err="failed to get container status \"0b4779504c7f195327c40dbccbadf373bbf36d2b85bb54ad92ea90834755a92e\": rpc error: code = NotFound desc = could not find container \"0b4779504c7f195327c40dbccbadf373bbf36d2b85bb54ad92ea90834755a92e\": container with ID starting with 0b4779504c7f195327c40dbccbadf373bbf36d2b85bb54ad92ea90834755a92e not found: ID does not exist"
Jul 02 13:49:20.899795 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:20.899761    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" path="/var/lib/kubelet/pods/ff4c6996-9d23-4dd8-b001-ed01cf58a717/volumes"
Jul 02 13:49:23.403899 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:23.403870    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:49:23.404433 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:23.404404    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr" podUID="27261a17-2625-4494-bd67-c89c19d41f32" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 02 13:49:33.405211 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:33.405163    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr" podUID="27261a17-2625-4494-bd67-c89c19d41f32" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 02 13:49:43.404619 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:43.404582    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr" podUID="27261a17-2625-4494-bd67-c89c19d41f32" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 02 13:49:49.914316 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:49.914279    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:50.049304 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.049270    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:50.075528 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.075498    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"]
Jul 02 13:49:50.075730 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.075705    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerName="kserve-container" containerID="cri-o://63ce6df5448a7c558b55c2195d6df4c3fc25cafb5c27212c45f3a6af92939e1e" gracePeriod=30
Jul 02 13:49:50.075878 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.075768    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerName="kube-rbac-proxy" containerID="cri-o://a02ea7e7a37b0ff1e1c1dd77a811974948c80759819215719e0b3aff965c3057" gracePeriod=30
Jul 02 13:49:50.100301 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.100273    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"]
Jul 02 13:49:50.100741 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.100723    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" containerName="kserve-container"
Jul 02 13:49:50.100832 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.100743    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" containerName="kserve-container"
Jul 02 13:49:50.100832 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.100755    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" containerName="kube-rbac-proxy"
Jul 02 13:49:50.100832 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.100762    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" containerName="kube-rbac-proxy"
Jul 02 13:49:50.100991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.100892    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" containerName="kube-rbac-proxy"
Jul 02 13:49:50.100991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.100907    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff4c6996-9d23-4dd8-b001-ed01cf58a717" containerName="kserve-container"
Jul 02 13:49:50.104199 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.104180    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 13:49:50.105949 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.105928    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-7f184-predictor-serving-cert\""
Jul 02 13:49:50.106058 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.106005    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-7f184-kube-rbac-proxy-sar-config\""
Jul 02 13:49:50.114893 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.114866    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"]
Jul 02 13:49:50.139323 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.139283    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:50.162574 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.162543    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1f204ab2-93ae-427e-ac6d-d436baa331e9-proxy-tls\") pod \"success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7\" (UID: \"1f204ab2-93ae-427e-ac6d-d436baa331e9\") " pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 13:49:50.162753 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.162624    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mllxm\" (UniqueName: \"kubernetes.io/projected/1f204ab2-93ae-427e-ac6d-d436baa331e9-kube-api-access-mllxm\") pod \"success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7\" (UID: \"1f204ab2-93ae-427e-ac6d-d436baa331e9\") " pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 13:49:50.162809 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.162759    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1f204ab2-93ae-427e-ac6d-d436baa331e9-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7\" (UID: \"1f204ab2-93ae-427e-ac6d-d436baa331e9\") " pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 13:49:50.236355 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.236312    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:50.263204 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.263167    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1f204ab2-93ae-427e-ac6d-d436baa331e9-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7\" (UID: \"1f204ab2-93ae-427e-ac6d-d436baa331e9\") " pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 13:49:50.263420 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.263224    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1f204ab2-93ae-427e-ac6d-d436baa331e9-proxy-tls\") pod \"success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7\" (UID: \"1f204ab2-93ae-427e-ac6d-d436baa331e9\") " pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 13:49:50.263420 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.263274    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mllxm\" (UniqueName: \"kubernetes.io/projected/1f204ab2-93ae-427e-ac6d-d436baa331e9-kube-api-access-mllxm\") pod \"success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7\" (UID: \"1f204ab2-93ae-427e-ac6d-d436baa331e9\") " pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 13:49:50.263553 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:49:50.263422    2570 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-7f184-predictor-serving-cert: secret "success-200-isvc-7f184-predictor-serving-cert" not found
Jul 02 13:49:50.263553 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:49:50.263507    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f204ab2-93ae-427e-ac6d-d436baa331e9-proxy-tls podName:1f204ab2-93ae-427e-ac6d-d436baa331e9 nodeName:}" failed. No retries permitted until 2026-07-02 13:49:50.763486906 +0000 UTC m=+2142.386279803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/1f204ab2-93ae-427e-ac6d-d436baa331e9-proxy-tls") pod "success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7" (UID: "1f204ab2-93ae-427e-ac6d-d436baa331e9") : secret "success-200-isvc-7f184-predictor-serving-cert" not found
Jul 02 13:49:50.263862 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.263843    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1f204ab2-93ae-427e-ac6d-d436baa331e9-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7\" (UID: \"1f204ab2-93ae-427e-ac6d-d436baa331e9\") " pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 13:49:50.272700 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.272674    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mllxm\" (UniqueName: \"kubernetes.io/projected/1f204ab2-93ae-427e-ac6d-d436baa331e9-kube-api-access-mllxm\") pod \"success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7\" (UID: \"1f204ab2-93ae-427e-ac6d-d436baa331e9\") " pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 13:49:50.554676 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.554633    2570 generic.go:356] "Generic (PLEG): container finished" podID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerID="a02ea7e7a37b0ff1e1c1dd77a811974948c80759819215719e0b3aff965c3057" exitCode=2
Jul 02 13:49:50.554859 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.554697    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" event={"ID":"f1a08288-183d-4a3d-8202-cd0b559e5a05","Type":"ContainerDied","Data":"a02ea7e7a37b0ff1e1c1dd77a811974948c80759819215719e0b3aff965c3057"}
Jul 02 13:49:50.587349 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.587316    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:50.688668 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.688634    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:49:50.767497 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.767461    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1f204ab2-93ae-427e-ac6d-d436baa331e9-proxy-tls\") pod \"success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7\" (UID: \"1f204ab2-93ae-427e-ac6d-d436baa331e9\") " pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 13:49:50.769934 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:50.769908    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1f204ab2-93ae-427e-ac6d-d436baa331e9-proxy-tls\") pod \"success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7\" (UID: \"1f204ab2-93ae-427e-ac6d-d436baa331e9\") " pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 13:49:51.017712 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:51.017672    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 13:49:51.148090 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:51.148065    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"]
Jul 02 13:49:51.149795 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:49:51.149767    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f204ab2_93ae_427e_ac6d_d436baa331e9.slice/crio-4727d87c45f33ea2c4aebe610c13c6bb567357110f5858969e92f82b75d86e1d WatchSource:0}: Error finding container 4727d87c45f33ea2c4aebe610c13c6bb567357110f5858969e92f82b75d86e1d: Status 404 returned error can't find the container with id 4727d87c45f33ea2c4aebe610c13c6bb567357110f5858969e92f82b75d86e1d
Jul 02 13:49:51.567475 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:51.567369    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7" event={"ID":"1f204ab2-93ae-427e-ac6d-d436baa331e9","Type":"ContainerStarted","Data":"cfd42b1191233babe28943ad57357b6c61e3d4141cd1ba7b855da3e6d4f29b76"}
Jul 02 13:49:51.567475 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:51.567471    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7" event={"ID":"1f204ab2-93ae-427e-ac6d-d436baa331e9","Type":"ContainerStarted","Data":"08e8150f598226d70bd23f2529167ea1b67d66ebf4019f2f5b2b2a26124eabe0"}
Jul 02 13:49:51.567670 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:51.567497    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 13:49:51.567670 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:51.567518    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7" event={"ID":"1f204ab2-93ae-427e-ac6d-d436baa331e9","Type":"ContainerStarted","Data":"4727d87c45f33ea2c4aebe610c13c6bb567357110f5858969e92f82b75d86e1d"}
Jul 02 13:49:51.581859 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:51.581809    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7" podStartSLOduration=1.581792338 podStartE2EDuration="1.581792338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:49:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:49:51.58052646 +0000 UTC m=+2143.203319377" watchObservedRunningTime="2026-07-02 13:49:51.581792338 +0000 UTC m=+2143.204585243"
Jul 02 13:49:52.572136 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:52.572100    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 13:49:52.573450 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:52.573417    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7" podUID="1f204ab2-93ae-427e-ac6d-d436baa331e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 02 13:49:52.931242 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:52.931149    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.40:8643/healthz\": dial tcp 10.132.0.40:8643: connect: connection refused"
Jul 02 13:49:52.935649 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:52.935610    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 02 13:49:53.405265 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.405223    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr" podUID="27261a17-2625-4494-bd67-c89c19d41f32" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 02 13:49:53.434881 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.434851    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:49:53.494825 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.494739    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6mmh\" (UniqueName: \"kubernetes.io/projected/f1a08288-183d-4a3d-8202-cd0b559e5a05-kube-api-access-f6mmh\") pod \"f1a08288-183d-4a3d-8202-cd0b559e5a05\" (UID: \"f1a08288-183d-4a3d-8202-cd0b559e5a05\") "
Jul 02 13:49:53.494825 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.494800    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f1a08288-183d-4a3d-8202-cd0b559e5a05-kube-rbac-proxy-sar-config\") pod \"f1a08288-183d-4a3d-8202-cd0b559e5a05\" (UID: \"f1a08288-183d-4a3d-8202-cd0b559e5a05\") "
Jul 02 13:49:53.495001 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.494837    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f1a08288-183d-4a3d-8202-cd0b559e5a05-proxy-tls\") pod \"f1a08288-183d-4a3d-8202-cd0b559e5a05\" (UID: \"f1a08288-183d-4a3d-8202-cd0b559e5a05\") "
Jul 02 13:49:53.495191 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.495158    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1a08288-183d-4a3d-8202-cd0b559e5a05-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f1a08288-183d-4a3d-8202-cd0b559e5a05" (UID: "f1a08288-183d-4a3d-8202-cd0b559e5a05"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:49:53.496878 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.496856    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1a08288-183d-4a3d-8202-cd0b559e5a05-kube-api-access-f6mmh" (OuterVolumeSpecName: "kube-api-access-f6mmh") pod "f1a08288-183d-4a3d-8202-cd0b559e5a05" (UID: "f1a08288-183d-4a3d-8202-cd0b559e5a05"). InnerVolumeSpecName "kube-api-access-f6mmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:49:53.496974 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.496944    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1a08288-183d-4a3d-8202-cd0b559e5a05-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f1a08288-183d-4a3d-8202-cd0b559e5a05" (UID: "f1a08288-183d-4a3d-8202-cd0b559e5a05"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:49:53.577800 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.577765    2570 generic.go:356] "Generic (PLEG): container finished" podID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerID="63ce6df5448a7c558b55c2195d6df4c3fc25cafb5c27212c45f3a6af92939e1e" exitCode=0
Jul 02 13:49:53.578308 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.577823    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"
Jul 02 13:49:53.578308 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.577852    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" event={"ID":"f1a08288-183d-4a3d-8202-cd0b559e5a05","Type":"ContainerDied","Data":"63ce6df5448a7c558b55c2195d6df4c3fc25cafb5c27212c45f3a6af92939e1e"}
Jul 02 13:49:53.578308 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.577905    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc" event={"ID":"f1a08288-183d-4a3d-8202-cd0b559e5a05","Type":"ContainerDied","Data":"ce948ff0b43496b82d54a69feedfb2e25b5e24d5ef939d009e71fdcea591ece6"}
Jul 02 13:49:53.578308 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.577924    2570 scope.go:117] "RemoveContainer" containerID="a02ea7e7a37b0ff1e1c1dd77a811974948c80759819215719e0b3aff965c3057"
Jul 02 13:49:53.578625 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.578429    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7" podUID="1f204ab2-93ae-427e-ac6d-d436baa331e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 02 13:49:53.587176 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.587160    2570 scope.go:117] "RemoveContainer" containerID="63ce6df5448a7c558b55c2195d6df4c3fc25cafb5c27212c45f3a6af92939e1e"
Jul 02 13:49:53.594775 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.594724    2570 scope.go:117] "RemoveContainer" containerID="a02ea7e7a37b0ff1e1c1dd77a811974948c80759819215719e0b3aff965c3057"
Jul 02 13:49:53.595219 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:49:53.595189    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a02ea7e7a37b0ff1e1c1dd77a811974948c80759819215719e0b3aff965c3057\": container with ID starting with a02ea7e7a37b0ff1e1c1dd77a811974948c80759819215719e0b3aff965c3057 not found: ID does not exist" containerID="a02ea7e7a37b0ff1e1c1dd77a811974948c80759819215719e0b3aff965c3057"
Jul 02 13:49:53.595324 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.595233    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a02ea7e7a37b0ff1e1c1dd77a811974948c80759819215719e0b3aff965c3057"} err="failed to get container status \"a02ea7e7a37b0ff1e1c1dd77a811974948c80759819215719e0b3aff965c3057\": rpc error: code = NotFound desc = could not find container \"a02ea7e7a37b0ff1e1c1dd77a811974948c80759819215719e0b3aff965c3057\": container with ID starting with a02ea7e7a37b0ff1e1c1dd77a811974948c80759819215719e0b3aff965c3057 not found: ID does not exist"
Jul 02 13:49:53.595324 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.595304    2570 scope.go:117] "RemoveContainer" containerID="63ce6df5448a7c558b55c2195d6df4c3fc25cafb5c27212c45f3a6af92939e1e"
Jul 02 13:49:53.595622 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.595603    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6mmh\" (UniqueName: \"kubernetes.io/projected/f1a08288-183d-4a3d-8202-cd0b559e5a05-kube-api-access-f6mmh\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:49:53.595701 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.595623    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f1a08288-183d-4a3d-8202-cd0b559e5a05-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:49:53.595701 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.595634    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f1a08288-183d-4a3d-8202-cd0b559e5a05-proxy-tls\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:49:53.595701 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:49:53.595644    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63ce6df5448a7c558b55c2195d6df4c3fc25cafb5c27212c45f3a6af92939e1e\": container with ID starting with 63ce6df5448a7c558b55c2195d6df4c3fc25cafb5c27212c45f3a6af92939e1e not found: ID does not exist" containerID="63ce6df5448a7c558b55c2195d6df4c3fc25cafb5c27212c45f3a6af92939e1e"
Jul 02 13:49:53.595701 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.595667    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63ce6df5448a7c558b55c2195d6df4c3fc25cafb5c27212c45f3a6af92939e1e"} err="failed to get container status \"63ce6df5448a7c558b55c2195d6df4c3fc25cafb5c27212c45f3a6af92939e1e\": rpc error: code = NotFound desc = could not find container \"63ce6df5448a7c558b55c2195d6df4c3fc25cafb5c27212c45f3a6af92939e1e\": container with ID starting with 63ce6df5448a7c558b55c2195d6df4c3fc25cafb5c27212c45f3a6af92939e1e not found: ID does not exist"
Jul 02 13:49:53.597169 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.597152    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"]
Jul 02 13:49:53.599350 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:53.599331    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7b15c-predictor-6cdf8ff9bf-rrrzc"]
Jul 02 13:49:54.897801 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:54.897769    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" path="/var/lib/kubelet/pods/f1a08288-183d-4a3d-8202-cd0b559e5a05/volumes"
Jul 02 13:49:58.584432 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:58.584379    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 13:49:58.584912 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:49:58.584883    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7" podUID="1f204ab2-93ae-427e-ac6d-d436baa331e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 02 13:50:03.405310 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:03.405227    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:50:08.585319 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:08.585274    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7" podUID="1f204ab2-93ae-427e-ac6d-d436baa331e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 02 13:50:15.477052 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:15.477022    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:16.019878 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:16.019844    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:18.585740 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:18.585701    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7" podUID="1f204ab2-93ae-427e-ac6d-d436baa331e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 02 13:50:25.572277 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.572196    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:25.653854 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.653819    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"]
Jul 02 13:50:25.654118 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.654091    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr" podUID="27261a17-2625-4494-bd67-c89c19d41f32" containerName="kserve-container" containerID="cri-o://3c34eab6fc551ec14d94c2e69d5e7476186fbd007189e2a2611aa3ae44130761" gracePeriod=30
Jul 02 13:50:25.654231 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.654142    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr" podUID="27261a17-2625-4494-bd67-c89c19d41f32" containerName="kube-rbac-proxy" containerID="cri-o://7ee5c2fd29ccf4a978a072d8533bd441a86163ea55d5c07cd0f19f71611e72b8" gracePeriod=30
Jul 02 13:50:25.663975 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.663948    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"]
Jul 02 13:50:25.664378 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.664364    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerName="kube-rbac-proxy"
Jul 02 13:50:25.664478 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.664378    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerName="kube-rbac-proxy"
Jul 02 13:50:25.664478 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.664456    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerName="kserve-container"
Jul 02 13:50:25.664478 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.664464    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerName="kserve-container"
Jul 02 13:50:25.664643 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.664540    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerName="kserve-container"
Jul 02 13:50:25.664643 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.664546    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1a08288-183d-4a3d-8202-cd0b559e5a05" containerName="kube-rbac-proxy"
Jul 02 13:50:25.668119 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.668101    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:50:25.670221 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.670194    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-03d0f-predictor-serving-cert\""
Jul 02 13:50:25.670307 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.670222    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-03d0f-kube-rbac-proxy-sar-config\""
Jul 02 13:50:25.679118 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.679092    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"]
Jul 02 13:50:25.710160 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.710130    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:25.767806 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.767763    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d83acafb-0f7a-479d-a051-9991dfabae58-proxy-tls\") pod \"success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm\" (UID: \"d83acafb-0f7a-479d-a051-9991dfabae58\") " pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:50:25.767806 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.767804    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d83acafb-0f7a-479d-a051-9991dfabae58-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm\" (UID: \"d83acafb-0f7a-479d-a051-9991dfabae58\") " pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:50:25.768055 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.767976    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqkcd\" (UniqueName: \"kubernetes.io/projected/d83acafb-0f7a-479d-a051-9991dfabae58-kube-api-access-bqkcd\") pod \"success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm\" (UID: \"d83acafb-0f7a-479d-a051-9991dfabae58\") " pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:50:25.775090 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.775069    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:25.834881 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.834783    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:25.868621 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.868585    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d83acafb-0f7a-479d-a051-9991dfabae58-proxy-tls\") pod \"success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm\" (UID: \"d83acafb-0f7a-479d-a051-9991dfabae58\") " pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:50:25.868621 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.868624    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d83acafb-0f7a-479d-a051-9991dfabae58-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm\" (UID: \"d83acafb-0f7a-479d-a051-9991dfabae58\") " pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:50:25.868865 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.868679    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqkcd\" (UniqueName: \"kubernetes.io/projected/d83acafb-0f7a-479d-a051-9991dfabae58-kube-api-access-bqkcd\") pod \"success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm\" (UID: \"d83acafb-0f7a-479d-a051-9991dfabae58\") " pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:50:25.869411 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.869371    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d83acafb-0f7a-479d-a051-9991dfabae58-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm\" (UID: \"d83acafb-0f7a-479d-a051-9991dfabae58\") " pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:50:25.871086 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.871069    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d83acafb-0f7a-479d-a051-9991dfabae58-proxy-tls\") pod \"success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm\" (UID: \"d83acafb-0f7a-479d-a051-9991dfabae58\") " pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:50:25.875449 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.875428    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqkcd\" (UniqueName: \"kubernetes.io/projected/d83acafb-0f7a-479d-a051-9991dfabae58-kube-api-access-bqkcd\") pod \"success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm\" (UID: \"d83acafb-0f7a-479d-a051-9991dfabae58\") " pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:50:25.981148 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:25.981105    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:50:26.014722 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:26.014693    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:26.117786 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:26.117759    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"]
Jul 02 13:50:26.119197 ip-10-0-131-238 kubenswrapper[2570]: W0702 13:50:26.119157    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd83acafb_0f7a_479d_a051_9991dfabae58.slice/crio-6687bc59f67cc316dbacd9ecdc23e3f50c7a3d61a4e92f1bb0742f7b8699cc80 WatchSource:0}: Error finding container 6687bc59f67cc316dbacd9ecdc23e3f50c7a3d61a4e92f1bb0742f7b8699cc80: Status 404 returned error can't find the container with id 6687bc59f67cc316dbacd9ecdc23e3f50c7a3d61a4e92f1bb0742f7b8699cc80
Jul 02 13:50:26.737232 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:26.737193    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm" event={"ID":"d83acafb-0f7a-479d-a051-9991dfabae58","Type":"ContainerStarted","Data":"50fc03c6d49868a293dc2ce5d700952bca3b90837da538f8ee20bde8037ee8ae"}
Jul 02 13:50:26.737709 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:26.737269    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm" event={"ID":"d83acafb-0f7a-479d-a051-9991dfabae58","Type":"ContainerStarted","Data":"0f3501e1619b75a2a5d0ff6e54a4eda67035ee2a3b0d16b7467f20d170236a99"}
Jul 02 13:50:26.737709 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:26.737298    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm" event={"ID":"d83acafb-0f7a-479d-a051-9991dfabae58","Type":"ContainerStarted","Data":"6687bc59f67cc316dbacd9ecdc23e3f50c7a3d61a4e92f1bb0742f7b8699cc80"}
Jul 02 13:50:26.737709 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:26.737327    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:50:26.739912 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:26.739881    2570 generic.go:356] "Generic (PLEG): container finished" podID="27261a17-2625-4494-bd67-c89c19d41f32" containerID="7ee5c2fd29ccf4a978a072d8533bd441a86163ea55d5c07cd0f19f71611e72b8" exitCode=2
Jul 02 13:50:26.740063 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:26.739951    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr" event={"ID":"27261a17-2625-4494-bd67-c89c19d41f32","Type":"ContainerDied","Data":"7ee5c2fd29ccf4a978a072d8533bd441a86163ea55d5c07cd0f19f71611e72b8"}
Jul 02 13:50:26.760097 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:26.760042    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm" podStartSLOduration=1.760022661 podStartE2EDuration="1.760022661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:50:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:50:26.758959314 +0000 UTC m=+2178.381752219" watchObservedRunningTime="2026-07-02 13:50:26.760022661 +0000 UTC m=+2178.382815568"
Jul 02 13:50:27.744933 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:27.744892    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:50:27.746160 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:27.746135    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm" podUID="d83acafb-0f7a-479d-a051-9991dfabae58" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 02 13:50:28.399288 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:28.399241    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr" podUID="27261a17-2625-4494-bd67-c89c19d41f32" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.41:8643/healthz\": dial tcp 10.132.0.41:8643: connect: connection refused"
Jul 02 13:50:28.584822 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:28.584785    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7" podUID="1f204ab2-93ae-427e-ac6d-d436baa331e9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 02 13:50:28.749791 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:28.749751    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm" podUID="d83acafb-0f7a-479d-a051-9991dfabae58" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 02 13:50:29.424565 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.424536    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:50:29.495601 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.495503    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/27261a17-2625-4494-bd67-c89c19d41f32-proxy-tls\") pod \"27261a17-2625-4494-bd67-c89c19d41f32\" (UID: \"27261a17-2625-4494-bd67-c89c19d41f32\") "
Jul 02 13:50:29.495601 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.495558    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6cdg\" (UniqueName: \"kubernetes.io/projected/27261a17-2625-4494-bd67-c89c19d41f32-kube-api-access-n6cdg\") pod \"27261a17-2625-4494-bd67-c89c19d41f32\" (UID: \"27261a17-2625-4494-bd67-c89c19d41f32\") "
Jul 02 13:50:29.495828 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.495617    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/27261a17-2625-4494-bd67-c89c19d41f32-kube-rbac-proxy-sar-config\") pod \"27261a17-2625-4494-bd67-c89c19d41f32\" (UID: \"27261a17-2625-4494-bd67-c89c19d41f32\") "
Jul 02 13:50:29.496048 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.496021    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27261a17-2625-4494-bd67-c89c19d41f32-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "27261a17-2625-4494-bd67-c89c19d41f32" (UID: "27261a17-2625-4494-bd67-c89c19d41f32"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:50:29.497610 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.497590    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27261a17-2625-4494-bd67-c89c19d41f32-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "27261a17-2625-4494-bd67-c89c19d41f32" (UID: "27261a17-2625-4494-bd67-c89c19d41f32"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:50:29.497685 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.497648    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27261a17-2625-4494-bd67-c89c19d41f32-kube-api-access-n6cdg" (OuterVolumeSpecName: "kube-api-access-n6cdg") pod "27261a17-2625-4494-bd67-c89c19d41f32" (UID: "27261a17-2625-4494-bd67-c89c19d41f32"). InnerVolumeSpecName "kube-api-access-n6cdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:50:29.597068 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.597029    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/27261a17-2625-4494-bd67-c89c19d41f32-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:50:29.597068 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.597063    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/27261a17-2625-4494-bd67-c89c19d41f32-proxy-tls\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:50:29.597068 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.597077    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6cdg\" (UniqueName: \"kubernetes.io/projected/27261a17-2625-4494-bd67-c89c19d41f32-kube-api-access-n6cdg\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:50:29.756509 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.756408    2570 generic.go:356] "Generic (PLEG): container finished" podID="27261a17-2625-4494-bd67-c89c19d41f32" containerID="3c34eab6fc551ec14d94c2e69d5e7476186fbd007189e2a2611aa3ae44130761" exitCode=0
Jul 02 13:50:29.756509 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.756472    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"
Jul 02 13:50:29.756509 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.756484    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr" event={"ID":"27261a17-2625-4494-bd67-c89c19d41f32","Type":"ContainerDied","Data":"3c34eab6fc551ec14d94c2e69d5e7476186fbd007189e2a2611aa3ae44130761"}
Jul 02 13:50:29.756991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.756546    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr" event={"ID":"27261a17-2625-4494-bd67-c89c19d41f32","Type":"ContainerDied","Data":"1cd803e9c93e47db792fac32ed102d39100bf52ca497bd4d392628682bacf405"}
Jul 02 13:50:29.756991 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.756576    2570 scope.go:117] "RemoveContainer" containerID="7ee5c2fd29ccf4a978a072d8533bd441a86163ea55d5c07cd0f19f71611e72b8"
Jul 02 13:50:29.765348 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.765331    2570 scope.go:117] "RemoveContainer" containerID="3c34eab6fc551ec14d94c2e69d5e7476186fbd007189e2a2611aa3ae44130761"
Jul 02 13:50:29.773396 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.773366    2570 scope.go:117] "RemoveContainer" containerID="7ee5c2fd29ccf4a978a072d8533bd441a86163ea55d5c07cd0f19f71611e72b8"
Jul 02 13:50:29.773651 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:50:29.773632    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ee5c2fd29ccf4a978a072d8533bd441a86163ea55d5c07cd0f19f71611e72b8\": container with ID starting with 7ee5c2fd29ccf4a978a072d8533bd441a86163ea55d5c07cd0f19f71611e72b8 not found: ID does not exist" containerID="7ee5c2fd29ccf4a978a072d8533bd441a86163ea55d5c07cd0f19f71611e72b8"
Jul 02 13:50:29.773700 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.773661    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ee5c2fd29ccf4a978a072d8533bd441a86163ea55d5c07cd0f19f71611e72b8"} err="failed to get container status \"7ee5c2fd29ccf4a978a072d8533bd441a86163ea55d5c07cd0f19f71611e72b8\": rpc error: code = NotFound desc = could not find container \"7ee5c2fd29ccf4a978a072d8533bd441a86163ea55d5c07cd0f19f71611e72b8\": container with ID starting with 7ee5c2fd29ccf4a978a072d8533bd441a86163ea55d5c07cd0f19f71611e72b8 not found: ID does not exist"
Jul 02 13:50:29.773700 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.773678    2570 scope.go:117] "RemoveContainer" containerID="3c34eab6fc551ec14d94c2e69d5e7476186fbd007189e2a2611aa3ae44130761"
Jul 02 13:50:29.773936 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:50:29.773910    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c34eab6fc551ec14d94c2e69d5e7476186fbd007189e2a2611aa3ae44130761\": container with ID starting with 3c34eab6fc551ec14d94c2e69d5e7476186fbd007189e2a2611aa3ae44130761 not found: ID does not exist" containerID="3c34eab6fc551ec14d94c2e69d5e7476186fbd007189e2a2611aa3ae44130761"
Jul 02 13:50:29.774627 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.773938    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c34eab6fc551ec14d94c2e69d5e7476186fbd007189e2a2611aa3ae44130761"} err="failed to get container status \"3c34eab6fc551ec14d94c2e69d5e7476186fbd007189e2a2611aa3ae44130761\": rpc error: code = NotFound desc = could not find container \"3c34eab6fc551ec14d94c2e69d5e7476186fbd007189e2a2611aa3ae44130761\": container with ID starting with 3c34eab6fc551ec14d94c2e69d5e7476186fbd007189e2a2611aa3ae44130761 not found: ID does not exist"
Jul 02 13:50:29.776194 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.776172    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"]
Jul 02 13:50:29.779217 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:29.779191    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9633b-predictor-774c89fdf5-kzkcr"]
Jul 02 13:50:30.897977 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:30.897933    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27261a17-2625-4494-bd67-c89c19d41f32" path="/var/lib/kubelet/pods/27261a17-2625-4494-bd67-c89c19d41f32/volumes"
Jul 02 13:50:33.754677 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:33.754647    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:50:33.755161 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:33.755134    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm" podUID="d83acafb-0f7a-479d-a051-9991dfabae58" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 02 13:50:38.586207 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:38.586177    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 13:50:43.756092 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:43.756050    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm" podUID="d83acafb-0f7a-479d-a051-9991dfabae58" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 02 13:50:50.119616 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:50.119566    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:50.379826 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:50.379743    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:50:53.755282 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:50:53.755239    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm" podUID="d83acafb-0f7a-479d-a051-9991dfabae58" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 02 13:51:03.755750 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:51:03.755715    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm" podUID="d83acafb-0f7a-479d-a051-9991dfabae58" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 02 13:51:13.755678 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:51:13.755647    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:51:25.803270 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:51:25.803239    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:51:25.922552 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:51:25.922510    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:53:19.547318 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:53:19.547287    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:54:09.050258 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:54:09.050223    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:54:09.056192 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:54:09.056170    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:54:09.056327 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:54:09.056253    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:54:09.062068 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:54:09.062045    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:57:04.322220 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:57:04.322184    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:59:09.081407 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:09.081299    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:59:09.087516 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:09.087493    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:59:09.088311 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:09.088292    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 13:59:09.093892 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:09.093876    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 13:59:40.414569 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:40.414524    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:59:40.485310 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:40.485279    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"]
Jul 02 13:59:40.485579 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:40.485528    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm" podUID="d83acafb-0f7a-479d-a051-9991dfabae58" containerName="kserve-container" containerID="cri-o://0f3501e1619b75a2a5d0ff6e54a4eda67035ee2a3b0d16b7467f20d170236a99" gracePeriod=30
Jul 02 13:59:40.485703 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:40.485546    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm" podUID="d83acafb-0f7a-479d-a051-9991dfabae58" containerName="kube-rbac-proxy" containerID="cri-o://50fc03c6d49868a293dc2ce5d700952bca3b90837da538f8ee20bde8037ee8ae" gracePeriod=30
Jul 02 13:59:41.346059 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:41.346016    2570 generic.go:356] "Generic (PLEG): container finished" podID="d83acafb-0f7a-479d-a051-9991dfabae58" containerID="50fc03c6d49868a293dc2ce5d700952bca3b90837da538f8ee20bde8037ee8ae" exitCode=2
Jul 02 13:59:41.346229 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:41.346094    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm" event={"ID":"d83acafb-0f7a-479d-a051-9991dfabae58","Type":"ContainerDied","Data":"50fc03c6d49868a293dc2ce5d700952bca3b90837da538f8ee20bde8037ee8ae"}
Jul 02 13:59:42.391168 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:42.391124    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 13:59:43.637244 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:43.637216    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:59:43.725812 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:43.725780    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d83acafb-0f7a-479d-a051-9991dfabae58-kube-rbac-proxy-sar-config\") pod \"d83acafb-0f7a-479d-a051-9991dfabae58\" (UID: \"d83acafb-0f7a-479d-a051-9991dfabae58\") "
Jul 02 13:59:43.725981 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:43.725865    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d83acafb-0f7a-479d-a051-9991dfabae58-proxy-tls\") pod \"d83acafb-0f7a-479d-a051-9991dfabae58\" (UID: \"d83acafb-0f7a-479d-a051-9991dfabae58\") "
Jul 02 13:59:43.725981 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:43.725888    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqkcd\" (UniqueName: \"kubernetes.io/projected/d83acafb-0f7a-479d-a051-9991dfabae58-kube-api-access-bqkcd\") pod \"d83acafb-0f7a-479d-a051-9991dfabae58\" (UID: \"d83acafb-0f7a-479d-a051-9991dfabae58\") "
Jul 02 13:59:43.726210 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:43.726185    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d83acafb-0f7a-479d-a051-9991dfabae58-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d83acafb-0f7a-479d-a051-9991dfabae58" (UID: "d83acafb-0f7a-479d-a051-9991dfabae58"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:59:43.727976 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:43.727953    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d83acafb-0f7a-479d-a051-9991dfabae58-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d83acafb-0f7a-479d-a051-9991dfabae58" (UID: "d83acafb-0f7a-479d-a051-9991dfabae58"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:59:43.728077 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:43.728009    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d83acafb-0f7a-479d-a051-9991dfabae58-kube-api-access-bqkcd" (OuterVolumeSpecName: "kube-api-access-bqkcd") pod "d83acafb-0f7a-479d-a051-9991dfabae58" (UID: "d83acafb-0f7a-479d-a051-9991dfabae58"). InnerVolumeSpecName "kube-api-access-bqkcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:59:43.827457 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:43.827416    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d83acafb-0f7a-479d-a051-9991dfabae58-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:59:43.827457 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:43.827450    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d83acafb-0f7a-479d-a051-9991dfabae58-proxy-tls\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:59:43.827457 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:43.827461    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqkcd\" (UniqueName: \"kubernetes.io/projected/d83acafb-0f7a-479d-a051-9991dfabae58-kube-api-access-bqkcd\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 13:59:44.367913 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:44.367878    2570 generic.go:356] "Generic (PLEG): container finished" podID="d83acafb-0f7a-479d-a051-9991dfabae58" containerID="0f3501e1619b75a2a5d0ff6e54a4eda67035ee2a3b0d16b7467f20d170236a99" exitCode=0
Jul 02 13:59:44.368131 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:44.367948    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"
Jul 02 13:59:44.368131 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:44.367951    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm" event={"ID":"d83acafb-0f7a-479d-a051-9991dfabae58","Type":"ContainerDied","Data":"0f3501e1619b75a2a5d0ff6e54a4eda67035ee2a3b0d16b7467f20d170236a99"}
Jul 02 13:59:44.368131 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:44.368097    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm" event={"ID":"d83acafb-0f7a-479d-a051-9991dfabae58","Type":"ContainerDied","Data":"6687bc59f67cc316dbacd9ecdc23e3f50c7a3d61a4e92f1bb0742f7b8699cc80"}
Jul 02 13:59:44.368131 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:44.368121    2570 scope.go:117] "RemoveContainer" containerID="50fc03c6d49868a293dc2ce5d700952bca3b90837da538f8ee20bde8037ee8ae"
Jul 02 13:59:44.377315 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:44.377298    2570 scope.go:117] "RemoveContainer" containerID="0f3501e1619b75a2a5d0ff6e54a4eda67035ee2a3b0d16b7467f20d170236a99"
Jul 02 13:59:44.384550 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:44.384534    2570 scope.go:117] "RemoveContainer" containerID="50fc03c6d49868a293dc2ce5d700952bca3b90837da538f8ee20bde8037ee8ae"
Jul 02 13:59:44.384762 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:59:44.384746    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50fc03c6d49868a293dc2ce5d700952bca3b90837da538f8ee20bde8037ee8ae\": container with ID starting with 50fc03c6d49868a293dc2ce5d700952bca3b90837da538f8ee20bde8037ee8ae not found: ID does not exist" containerID="50fc03c6d49868a293dc2ce5d700952bca3b90837da538f8ee20bde8037ee8ae"
Jul 02 13:59:44.384816 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:44.384768    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50fc03c6d49868a293dc2ce5d700952bca3b90837da538f8ee20bde8037ee8ae"} err="failed to get container status \"50fc03c6d49868a293dc2ce5d700952bca3b90837da538f8ee20bde8037ee8ae\": rpc error: code = NotFound desc = could not find container \"50fc03c6d49868a293dc2ce5d700952bca3b90837da538f8ee20bde8037ee8ae\": container with ID starting with 50fc03c6d49868a293dc2ce5d700952bca3b90837da538f8ee20bde8037ee8ae not found: ID does not exist"
Jul 02 13:59:44.384816 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:44.384783    2570 scope.go:117] "RemoveContainer" containerID="0f3501e1619b75a2a5d0ff6e54a4eda67035ee2a3b0d16b7467f20d170236a99"
Jul 02 13:59:44.385004 ip-10-0-131-238 kubenswrapper[2570]: E0702 13:59:44.384984    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f3501e1619b75a2a5d0ff6e54a4eda67035ee2a3b0d16b7467f20d170236a99\": container with ID starting with 0f3501e1619b75a2a5d0ff6e54a4eda67035ee2a3b0d16b7467f20d170236a99 not found: ID does not exist" containerID="0f3501e1619b75a2a5d0ff6e54a4eda67035ee2a3b0d16b7467f20d170236a99"
Jul 02 13:59:44.385059 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:44.385014    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f3501e1619b75a2a5d0ff6e54a4eda67035ee2a3b0d16b7467f20d170236a99"} err="failed to get container status \"0f3501e1619b75a2a5d0ff6e54a4eda67035ee2a3b0d16b7467f20d170236a99\": rpc error: code = NotFound desc = could not find container \"0f3501e1619b75a2a5d0ff6e54a4eda67035ee2a3b0d16b7467f20d170236a99\": container with ID starting with 0f3501e1619b75a2a5d0ff6e54a4eda67035ee2a3b0d16b7467f20d170236a99 not found: ID does not exist"
Jul 02 13:59:44.387560 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:44.387540    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"]
Jul 02 13:59:44.390629 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:44.390611    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-03d0f-predictor-75bbcf79f5-hs9hm"]
Jul 02 13:59:44.898584 ip-10-0-131-238 kubenswrapper[2570]: I0702 13:59:44.898549    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d83acafb-0f7a-479d-a051-9991dfabae58" path="/var/lib/kubelet/pods/d83acafb-0f7a-479d-a051-9991dfabae58/volumes"
Jul 02 14:00:16.622917 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:00:16.622882    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 14:04:09.113138 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:04:09.113023    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 14:04:09.119642 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:04:09.119621    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 14:04:09.122003 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:04:09.121969    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 14:04:09.131203 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:04:09.131178    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 14:04:23.264292 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:04:23.264263    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 14:07:09.494749 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:09.494716    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 14:07:09.587679 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:09.587637    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"]
Jul 02 14:07:09.587905 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:09.587861    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7" podUID="1f204ab2-93ae-427e-ac6d-d436baa331e9" containerName="kserve-container" containerID="cri-o://08e8150f598226d70bd23f2529167ea1b67d66ebf4019f2f5b2b2a26124eabe0" gracePeriod=30
Jul 02 14:07:09.588003 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:09.587946    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7" podUID="1f204ab2-93ae-427e-ac6d-d436baa331e9" containerName="kube-rbac-proxy" containerID="cri-o://cfd42b1191233babe28943ad57357b6c61e3d4141cd1ba7b855da3e6d4f29b76" gracePeriod=30
Jul 02 14:07:10.448284 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.448252    2570 generic.go:356] "Generic (PLEG): container finished" podID="1f204ab2-93ae-427e-ac6d-d436baa331e9" containerID="cfd42b1191233babe28943ad57357b6c61e3d4141cd1ba7b855da3e6d4f29b76" exitCode=2
Jul 02 14:07:10.448484 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.448322    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7" event={"ID":"1f204ab2-93ae-427e-ac6d-d436baa331e9","Type":"ContainerDied","Data":"cfd42b1191233babe28943ad57357b6c61e3d4141cd1ba7b855da3e6d4f29b76"}
Jul 02 14:07:10.700328 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.700245    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ls8ll/must-gather-7xfsj"]
Jul 02 14:07:10.700694 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.700646    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27261a17-2625-4494-bd67-c89c19d41f32" containerName="kube-rbac-proxy"
Jul 02 14:07:10.700694 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.700658    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="27261a17-2625-4494-bd67-c89c19d41f32" containerName="kube-rbac-proxy"
Jul 02 14:07:10.700694 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.700670    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27261a17-2625-4494-bd67-c89c19d41f32" containerName="kserve-container"
Jul 02 14:07:10.700694 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.700676    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="27261a17-2625-4494-bd67-c89c19d41f32" containerName="kserve-container"
Jul 02 14:07:10.700694 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.700693    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d83acafb-0f7a-479d-a051-9991dfabae58" containerName="kube-rbac-proxy"
Jul 02 14:07:10.700894 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.700699    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d83acafb-0f7a-479d-a051-9991dfabae58" containerName="kube-rbac-proxy"
Jul 02 14:07:10.700894 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.700709    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d83acafb-0f7a-479d-a051-9991dfabae58" containerName="kserve-container"
Jul 02 14:07:10.700894 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.700714    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d83acafb-0f7a-479d-a051-9991dfabae58" containerName="kserve-container"
Jul 02 14:07:10.700894 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.700768    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d83acafb-0f7a-479d-a051-9991dfabae58" containerName="kserve-container"
Jul 02 14:07:10.700894 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.700777    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="27261a17-2625-4494-bd67-c89c19d41f32" containerName="kserve-container"
Jul 02 14:07:10.700894 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.700782    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d83acafb-0f7a-479d-a051-9991dfabae58" containerName="kube-rbac-proxy"
Jul 02 14:07:10.700894 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.700790    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="27261a17-2625-4494-bd67-c89c19d41f32" containerName="kube-rbac-proxy"
Jul 02 14:07:10.703853 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.703837    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ls8ll/must-gather-7xfsj"
Jul 02 14:07:10.705600 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.705572    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ls8ll\"/\"default-dockercfg-jbd57\""
Jul 02 14:07:10.705600 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.705572    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ls8ll\"/\"openshift-service-ca.crt\""
Jul 02 14:07:10.706096 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.706078    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ls8ll\"/\"kube-root-ca.crt\""
Jul 02 14:07:10.713874 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.713853    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ls8ll/must-gather-7xfsj"]
Jul 02 14:07:10.837288 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.837247    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/33d3477d-bffc-4436-8318-1c005caac047-must-gather-output\") pod \"must-gather-7xfsj\" (UID: \"33d3477d-bffc-4436-8318-1c005caac047\") " pod="openshift-must-gather-ls8ll/must-gather-7xfsj"
Jul 02 14:07:10.837512 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.837298    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2j9z\" (UniqueName: \"kubernetes.io/projected/33d3477d-bffc-4436-8318-1c005caac047-kube-api-access-g2j9z\") pod \"must-gather-7xfsj\" (UID: \"33d3477d-bffc-4436-8318-1c005caac047\") " pod="openshift-must-gather-ls8ll/must-gather-7xfsj"
Jul 02 14:07:10.938788 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.938749    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/33d3477d-bffc-4436-8318-1c005caac047-must-gather-output\") pod \"must-gather-7xfsj\" (UID: \"33d3477d-bffc-4436-8318-1c005caac047\") " pod="openshift-must-gather-ls8ll/must-gather-7xfsj"
Jul 02 14:07:10.938964 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.938798    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2j9z\" (UniqueName: \"kubernetes.io/projected/33d3477d-bffc-4436-8318-1c005caac047-kube-api-access-g2j9z\") pod \"must-gather-7xfsj\" (UID: \"33d3477d-bffc-4436-8318-1c005caac047\") " pod="openshift-must-gather-ls8ll/must-gather-7xfsj"
Jul 02 14:07:10.939171 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.939147    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/33d3477d-bffc-4436-8318-1c005caac047-must-gather-output\") pod \"must-gather-7xfsj\" (UID: \"33d3477d-bffc-4436-8318-1c005caac047\") " pod="openshift-must-gather-ls8ll/must-gather-7xfsj"
Jul 02 14:07:10.945539 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:10.945514    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2j9z\" (UniqueName: \"kubernetes.io/projected/33d3477d-bffc-4436-8318-1c005caac047-kube-api-access-g2j9z\") pod \"must-gather-7xfsj\" (UID: \"33d3477d-bffc-4436-8318-1c005caac047\") " pod="openshift-must-gather-ls8ll/must-gather-7xfsj"
Jul 02 14:07:11.023021 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:11.022931    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ls8ll/must-gather-7xfsj"
Jul 02 14:07:11.158266 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:11.153809    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ls8ll/must-gather-7xfsj"]
Jul 02 14:07:11.159574 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:11.159551    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 14:07:11.434202 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:11.434115    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 14:07:11.454286 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:11.454249    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ls8ll/must-gather-7xfsj" event={"ID":"33d3477d-bffc-4436-8318-1c005caac047","Type":"ContainerStarted","Data":"b79d3098f218319d1c82ab4d3c927a6a25590c312e63161ec929aa5ab6bc3ee6"}
Jul 02 14:07:13.092802 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.092720    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 14:07:13.261788 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.261749    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1f204ab2-93ae-427e-ac6d-d436baa331e9-proxy-tls\") pod \"1f204ab2-93ae-427e-ac6d-d436baa331e9\" (UID: \"1f204ab2-93ae-427e-ac6d-d436baa331e9\") "
Jul 02 14:07:13.261979 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.261809    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1f204ab2-93ae-427e-ac6d-d436baa331e9-kube-rbac-proxy-sar-config\") pod \"1f204ab2-93ae-427e-ac6d-d436baa331e9\" (UID: \"1f204ab2-93ae-427e-ac6d-d436baa331e9\") "
Jul 02 14:07:13.261979 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.261889    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mllxm\" (UniqueName: \"kubernetes.io/projected/1f204ab2-93ae-427e-ac6d-d436baa331e9-kube-api-access-mllxm\") pod \"1f204ab2-93ae-427e-ac6d-d436baa331e9\" (UID: \"1f204ab2-93ae-427e-ac6d-d436baa331e9\") "
Jul 02 14:07:13.262518 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.262487    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f204ab2-93ae-427e-ac6d-d436baa331e9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1f204ab2-93ae-427e-ac6d-d436baa331e9" (UID: "1f204ab2-93ae-427e-ac6d-d436baa331e9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:07:13.264634 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.264608    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f204ab2-93ae-427e-ac6d-d436baa331e9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1f204ab2-93ae-427e-ac6d-d436baa331e9" (UID: "1f204ab2-93ae-427e-ac6d-d436baa331e9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:07:13.264787 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.264759    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f204ab2-93ae-427e-ac6d-d436baa331e9-kube-api-access-mllxm" (OuterVolumeSpecName: "kube-api-access-mllxm") pod "1f204ab2-93ae-427e-ac6d-d436baa331e9" (UID: "1f204ab2-93ae-427e-ac6d-d436baa331e9"). InnerVolumeSpecName "kube-api-access-mllxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:07:13.363513 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.363417    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1f204ab2-93ae-427e-ac6d-d436baa331e9-proxy-tls\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 14:07:13.363513 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.363473    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1f204ab2-93ae-427e-ac6d-d436baa331e9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 14:07:13.363513 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.363490    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mllxm\" (UniqueName: \"kubernetes.io/projected/1f204ab2-93ae-427e-ac6d-d436baa331e9-kube-api-access-mllxm\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 14:07:13.467310 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.467214    2570 generic.go:356] "Generic (PLEG): container finished" podID="1f204ab2-93ae-427e-ac6d-d436baa331e9" containerID="08e8150f598226d70bd23f2529167ea1b67d66ebf4019f2f5b2b2a26124eabe0" exitCode=0
Jul 02 14:07:13.467310 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.467280    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"
Jul 02 14:07:13.467310 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.467297    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7" event={"ID":"1f204ab2-93ae-427e-ac6d-d436baa331e9","Type":"ContainerDied","Data":"08e8150f598226d70bd23f2529167ea1b67d66ebf4019f2f5b2b2a26124eabe0"}
Jul 02 14:07:13.467635 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.467356    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7" event={"ID":"1f204ab2-93ae-427e-ac6d-d436baa331e9","Type":"ContainerDied","Data":"4727d87c45f33ea2c4aebe610c13c6bb567357110f5858969e92f82b75d86e1d"}
Jul 02 14:07:13.467635 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.467414    2570 scope.go:117] "RemoveContainer" containerID="cfd42b1191233babe28943ad57357b6c61e3d4141cd1ba7b855da3e6d4f29b76"
Jul 02 14:07:13.489445 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.489327    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"]
Jul 02 14:07:13.491241 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:13.491212    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7f184-predictor-5f7fcfbcbf-747l7"]
Jul 02 14:07:14.897635 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:14.897598    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f204ab2-93ae-427e-ac6d-d436baa331e9" path="/var/lib/kubelet/pods/1f204ab2-93ae-427e-ac6d-d436baa331e9/volumes"
Jul 02 14:07:16.309768 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:16.309609    2570 scope.go:117] "RemoveContainer" containerID="08e8150f598226d70bd23f2529167ea1b67d66ebf4019f2f5b2b2a26124eabe0"
Jul 02 14:07:17.209005 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:17.208980    2570 scope.go:117] "RemoveContainer" containerID="cfd42b1191233babe28943ad57357b6c61e3d4141cd1ba7b855da3e6d4f29b76"
Jul 02 14:07:17.209307 ip-10-0-131-238 kubenswrapper[2570]: E0702 14:07:17.209288    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfd42b1191233babe28943ad57357b6c61e3d4141cd1ba7b855da3e6d4f29b76\": container with ID starting with cfd42b1191233babe28943ad57357b6c61e3d4141cd1ba7b855da3e6d4f29b76 not found: ID does not exist" containerID="cfd42b1191233babe28943ad57357b6c61e3d4141cd1ba7b855da3e6d4f29b76"
Jul 02 14:07:17.209349 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:17.209318    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfd42b1191233babe28943ad57357b6c61e3d4141cd1ba7b855da3e6d4f29b76"} err="failed to get container status \"cfd42b1191233babe28943ad57357b6c61e3d4141cd1ba7b855da3e6d4f29b76\": rpc error: code = NotFound desc = could not find container \"cfd42b1191233babe28943ad57357b6c61e3d4141cd1ba7b855da3e6d4f29b76\": container with ID starting with cfd42b1191233babe28943ad57357b6c61e3d4141cd1ba7b855da3e6d4f29b76 not found: ID does not exist"
Jul 02 14:07:17.209349 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:17.209337    2570 scope.go:117] "RemoveContainer" containerID="08e8150f598226d70bd23f2529167ea1b67d66ebf4019f2f5b2b2a26124eabe0"
Jul 02 14:07:17.209642 ip-10-0-131-238 kubenswrapper[2570]: E0702 14:07:17.209621    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08e8150f598226d70bd23f2529167ea1b67d66ebf4019f2f5b2b2a26124eabe0\": container with ID starting with 08e8150f598226d70bd23f2529167ea1b67d66ebf4019f2f5b2b2a26124eabe0 not found: ID does not exist" containerID="08e8150f598226d70bd23f2529167ea1b67d66ebf4019f2f5b2b2a26124eabe0"
Jul 02 14:07:17.209706 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:17.209653    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08e8150f598226d70bd23f2529167ea1b67d66ebf4019f2f5b2b2a26124eabe0"} err="failed to get container status \"08e8150f598226d70bd23f2529167ea1b67d66ebf4019f2f5b2b2a26124eabe0\": rpc error: code = NotFound desc = could not find container \"08e8150f598226d70bd23f2529167ea1b67d66ebf4019f2f5b2b2a26124eabe0\": container with ID starting with 08e8150f598226d70bd23f2529167ea1b67d66ebf4019f2f5b2b2a26124eabe0 not found: ID does not exist"
Jul 02 14:07:17.494094 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:17.493900    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ls8ll/must-gather-7xfsj" event={"ID":"33d3477d-bffc-4436-8318-1c005caac047","Type":"ContainerStarted","Data":"59d261de16f4ffb6282c4edda8afb8835fc90d5421d9422c0625045460346fbf"}
Jul 02 14:07:18.502047 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:18.502012    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ls8ll/must-gather-7xfsj" event={"ID":"33d3477d-bffc-4436-8318-1c005caac047","Type":"ContainerStarted","Data":"3861dc27c99e2d80639b8a3624039c8c298489af464e976ce23e88c259246a99"}
Jul 02 14:07:18.517049 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:18.516988    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ls8ll/must-gather-7xfsj" podStartSLOduration=2.356459567 podStartE2EDuration="8.51696676s" totalImagesPullingTime="6.160507193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:07:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:07:18.51451006 +0000 UTC m=+3190.137302968" watchObservedRunningTime="2026-07-02 14:07:18.51696676 +0000 UTC m=+3190.139759667"
Jul 02 14:07:39.612797 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:39.612713    2570 generic.go:356] "Generic (PLEG): container finished" podID="33d3477d-bffc-4436-8318-1c005caac047" containerID="59d261de16f4ffb6282c4edda8afb8835fc90d5421d9422c0625045460346fbf" exitCode=0
Jul 02 14:07:39.612797 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:39.612785    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ls8ll/must-gather-7xfsj" event={"ID":"33d3477d-bffc-4436-8318-1c005caac047","Type":"ContainerDied","Data":"59d261de16f4ffb6282c4edda8afb8835fc90d5421d9422c0625045460346fbf"}
Jul 02 14:07:39.613243 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:39.613105    2570 scope.go:117] "RemoveContainer" containerID="59d261de16f4ffb6282c4edda8afb8835fc90d5421d9422c0625045460346fbf"
Jul 02 14:07:40.244174 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:40.244147    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-ls8ll_must-gather-7xfsj_33d3477d-bffc-4436-8318-1c005caac047/gather/0.log"
Jul 02 14:07:40.679764 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:40.679689    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 14:07:40.836079 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:40.836044    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 14:07:40.891704 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:40.891664    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 14:07:43.602438 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:43.602400    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bhrhl_a8a9f309-636c-41da-a9a7-6d33ebced403/global-pull-secret-syncer/0.log"
Jul 02 14:07:43.688966 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:43.688937    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log"
Jul 02 14:07:43.767496 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:43.767462    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-238.ec2.internal_10fd38b2c4f9fdad3bb40dd53988d42d/haproxy/0.log"
Jul 02 14:07:45.667860 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:45.667828    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-ls8ll/must-gather-7xfsj"]
Jul 02 14:07:45.668408 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:45.668047    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-ls8ll/must-gather-7xfsj" podUID="33d3477d-bffc-4436-8318-1c005caac047" containerName="copy" containerID="cri-o://3861dc27c99e2d80639b8a3624039c8c298489af464e976ce23e88c259246a99" gracePeriod=2
Jul 02 14:07:45.669446 ip-10-0-131-238 kubenswrapper[2570]: E0702 14:07:45.669417    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-7xfsj\" is forbidden: User \"system:node:ip-10-0-131-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-ls8ll\": no relationship found between node 'ip-10-0-131-238.ec2.internal' and this object" podUID="33d3477d-bffc-4436-8318-1c005caac047" pod="openshift-must-gather-ls8ll/must-gather-7xfsj"
Jul 02 14:07:45.669782 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:45.669763    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-ls8ll/must-gather-7xfsj"]
Jul 02 14:07:45.902100 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:45.902074    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-ls8ll_must-gather-7xfsj_33d3477d-bffc-4436-8318-1c005caac047/copy/0.log"
Jul 02 14:07:45.904554 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:45.904527    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ls8ll/must-gather-7xfsj"
Jul 02 14:07:45.905989 ip-10-0-131-238 kubenswrapper[2570]: E0702 14:07:45.905962    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-7xfsj\" is forbidden: User \"system:node:ip-10-0-131-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-ls8ll\": no relationship found between node 'ip-10-0-131-238.ec2.internal' and this object" podUID="33d3477d-bffc-4436-8318-1c005caac047" pod="openshift-must-gather-ls8ll/must-gather-7xfsj"
Jul 02 14:07:46.058711 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.058669    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/33d3477d-bffc-4436-8318-1c005caac047-must-gather-output\") pod \"33d3477d-bffc-4436-8318-1c005caac047\" (UID: \"33d3477d-bffc-4436-8318-1c005caac047\") "
Jul 02 14:07:46.058912 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.058739    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2j9z\" (UniqueName: \"kubernetes.io/projected/33d3477d-bffc-4436-8318-1c005caac047-kube-api-access-g2j9z\") pod \"33d3477d-bffc-4436-8318-1c005caac047\" (UID: \"33d3477d-bffc-4436-8318-1c005caac047\") "
Jul 02 14:07:46.060231 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.060208    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33d3477d-bffc-4436-8318-1c005caac047-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "33d3477d-bffc-4436-8318-1c005caac047" (UID: "33d3477d-bffc-4436-8318-1c005caac047"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:07:46.061002 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.060978    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33d3477d-bffc-4436-8318-1c005caac047-kube-api-access-g2j9z" (OuterVolumeSpecName: "kube-api-access-g2j9z") pod "33d3477d-bffc-4436-8318-1c005caac047" (UID: "33d3477d-bffc-4436-8318-1c005caac047"). InnerVolumeSpecName "kube-api-access-g2j9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:07:46.159324 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.159284    2570 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/33d3477d-bffc-4436-8318-1c005caac047-must-gather-output\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 14:07:46.159324 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.159318    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2j9z\" (UniqueName: \"kubernetes.io/projected/33d3477d-bffc-4436-8318-1c005caac047-kube-api-access-g2j9z\") on node \"ip-10-0-131-238.ec2.internal\" DevicePath \"\""
Jul 02 14:07:46.277075 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.277038    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 14:07:46.328152 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.328068    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 14:07:46.652281 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.652204    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-ls8ll_must-gather-7xfsj_33d3477d-bffc-4436-8318-1c005caac047/copy/0.log"
Jul 02 14:07:46.652575 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.652549    2570 generic.go:356] "Generic (PLEG): container finished" podID="33d3477d-bffc-4436-8318-1c005caac047" containerID="3861dc27c99e2d80639b8a3624039c8c298489af464e976ce23e88c259246a99" exitCode=143
Jul 02 14:07:46.652639 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.652601    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ls8ll/must-gather-7xfsj"
Jul 02 14:07:46.652677 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.652645    2570 scope.go:117] "RemoveContainer" containerID="3861dc27c99e2d80639b8a3624039c8c298489af464e976ce23e88c259246a99"
Jul 02 14:07:46.655196 ip-10-0-131-238 kubenswrapper[2570]: E0702 14:07:46.655174    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-7xfsj\" is forbidden: User \"system:node:ip-10-0-131-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-ls8ll\": no relationship found between node 'ip-10-0-131-238.ec2.internal' and this object" podUID="33d3477d-bffc-4436-8318-1c005caac047" pod="openshift-must-gather-ls8ll/must-gather-7xfsj"
Jul 02 14:07:46.660736 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.660721    2570 scope.go:117] "RemoveContainer" containerID="59d261de16f4ffb6282c4edda8afb8835fc90d5421d9422c0625045460346fbf"
Jul 02 14:07:46.661832 ip-10-0-131-238 kubenswrapper[2570]: E0702 14:07:46.661804    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-7xfsj\" is forbidden: User \"system:node:ip-10-0-131-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-ls8ll\": no relationship found between node 'ip-10-0-131-238.ec2.internal' and this object" podUID="33d3477d-bffc-4436-8318-1c005caac047" pod="openshift-must-gather-ls8ll/must-gather-7xfsj"
Jul 02 14:07:46.673613 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.673482    2570 scope.go:117] "RemoveContainer" containerID="3861dc27c99e2d80639b8a3624039c8c298489af464e976ce23e88c259246a99"
Jul 02 14:07:46.673816 ip-10-0-131-238 kubenswrapper[2570]: E0702 14:07:46.673739    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3861dc27c99e2d80639b8a3624039c8c298489af464e976ce23e88c259246a99\": container with ID starting with 3861dc27c99e2d80639b8a3624039c8c298489af464e976ce23e88c259246a99 not found: ID does not exist" containerID="3861dc27c99e2d80639b8a3624039c8c298489af464e976ce23e88c259246a99"
Jul 02 14:07:46.673816 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.673766    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3861dc27c99e2d80639b8a3624039c8c298489af464e976ce23e88c259246a99"} err="failed to get container status \"3861dc27c99e2d80639b8a3624039c8c298489af464e976ce23e88c259246a99\": rpc error: code = NotFound desc = could not find container \"3861dc27c99e2d80639b8a3624039c8c298489af464e976ce23e88c259246a99\": container with ID starting with 3861dc27c99e2d80639b8a3624039c8c298489af464e976ce23e88c259246a99 not found: ID does not exist"
Jul 02 14:07:46.673816 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.673783    2570 scope.go:117] "RemoveContainer" containerID="59d261de16f4ffb6282c4edda8afb8835fc90d5421d9422c0625045460346fbf"
Jul 02 14:07:46.674016 ip-10-0-131-238 kubenswrapper[2570]: E0702 14:07:46.673999    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59d261de16f4ffb6282c4edda8afb8835fc90d5421d9422c0625045460346fbf\": container with ID starting with 59d261de16f4ffb6282c4edda8afb8835fc90d5421d9422c0625045460346fbf not found: ID does not exist" containerID="59d261de16f4ffb6282c4edda8afb8835fc90d5421d9422c0625045460346fbf"
Jul 02 14:07:46.674055 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.674021    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59d261de16f4ffb6282c4edda8afb8835fc90d5421d9422c0625045460346fbf"} err="failed to get container status \"59d261de16f4ffb6282c4edda8afb8835fc90d5421d9422c0625045460346fbf\": rpc error: code = NotFound desc = could not find container \"59d261de16f4ffb6282c4edda8afb8835fc90d5421d9422c0625045460346fbf\": container with ID starting with 59d261de16f4ffb6282c4edda8afb8835fc90d5421d9422c0625045460346fbf not found: ID does not exist"
Jul 02 14:07:46.897450 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:46.897416    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33d3477d-bffc-4436-8318-1c005caac047" path="/var/lib/kubelet/pods/33d3477d-bffc-4436-8318-1c005caac047/volumes"
Jul 02 14:07:48.372972 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:48.372942    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9749c51e-14be-4fa9-8122-00ffe30aa2ca/alertmanager/0.log"
Jul 02 14:07:48.393042 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:48.393019    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9749c51e-14be-4fa9-8122-00ffe30aa2ca/config-reloader/0.log"
Jul 02 14:07:48.418147 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:48.418121    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9749c51e-14be-4fa9-8122-00ffe30aa2ca/kube-rbac-proxy-web/0.log"
Jul 02 14:07:48.443902 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:48.443879    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9749c51e-14be-4fa9-8122-00ffe30aa2ca/kube-rbac-proxy/0.log"
Jul 02 14:07:48.462554 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:48.462528    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9749c51e-14be-4fa9-8122-00ffe30aa2ca/kube-rbac-proxy-metric/0.log"
Jul 02 14:07:48.482258 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:48.482236    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9749c51e-14be-4fa9-8122-00ffe30aa2ca/prom-label-proxy/0.log"
Jul 02 14:07:48.499490 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:48.499471    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9749c51e-14be-4fa9-8122-00ffe30aa2ca/init-config-reloader/0.log"
Jul 02 14:07:48.537571 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:48.537528    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/1.log"
Jul 02 14:07:48.611538 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:48.611503    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-z4gzj_ed6ac0b5-82f0-4681-ab35-abe5a4a50d80/cluster-monitoring-operator/0.log"
Jul 02 14:07:48.723564 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:48.723536    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-cd8656b8-7czq5_fd90139e-accb-4666-ac4d-dce1fcbf1cb4/monitoring-plugin/0.log"
Jul 02 14:07:48.824251 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:48.824225    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mpvj8_c605dce2-9e15-479c-a2bf-30cd447ccc0d/node-exporter/0.log"
Jul 02 14:07:48.852243 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:48.852212    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mpvj8_c605dce2-9e15-479c-a2bf-30cd447ccc0d/kube-rbac-proxy/0.log"
Jul 02 14:07:48.870529 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:48.870502    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mpvj8_c605dce2-9e15-479c-a2bf-30cd447ccc0d/init-textfile/0.log"
Jul 02 14:07:49.176006 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:49.175922    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-qzk2p_9ae30b20-eccb-40a4-a74c-4ad792f93d8a/prometheus-operator/0.log"
Jul 02 14:07:49.191605 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:49.191579    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-qzk2p_9ae30b20-eccb-40a4-a74c-4ad792f93d8a/kube-rbac-proxy/0.log"
Jul 02 14:07:49.214156 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:49.214135    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7ddc8784d4-jxkcp_d87a4a62-6245-4a66-a3d2-ba1f70555c1d/prometheus-operator-admission-webhook/0.log"
Jul 02 14:07:49.245139 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:49.245103    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-675f768f9b-89m2k_c07a06ee-5118-4201-8a60-b2d7ec5270c0/telemeter-client/0.log"
Jul 02 14:07:49.265132 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:49.265112    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-675f768f9b-89m2k_c07a06ee-5118-4201-8a60-b2d7ec5270c0/reload/0.log"
Jul 02 14:07:49.291150 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:49.291127    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-675f768f9b-89m2k_c07a06ee-5118-4201-8a60-b2d7ec5270c0/kube-rbac-proxy/0.log"
Jul 02 14:07:49.317258 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:49.317236    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6597fcffbc-wmh4k_8a02d5b3-ea9e-4f73-b2d7-527250090419/thanos-query/0.log"
Jul 02 14:07:49.333429 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:49.333409    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6597fcffbc-wmh4k_8a02d5b3-ea9e-4f73-b2d7-527250090419/kube-rbac-proxy-web/0.log"
Jul 02 14:07:49.349415 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:49.349374    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6597fcffbc-wmh4k_8a02d5b3-ea9e-4f73-b2d7-527250090419/kube-rbac-proxy/0.log"
Jul 02 14:07:49.366796 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:49.366778    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6597fcffbc-wmh4k_8a02d5b3-ea9e-4f73-b2d7-527250090419/prom-label-proxy/0.log"
Jul 02 14:07:49.385531 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:49.385514    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6597fcffbc-wmh4k_8a02d5b3-ea9e-4f73-b2d7-527250090419/kube-rbac-proxy-rules/0.log"
Jul 02 14:07:49.405349 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:49.405333    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6597fcffbc-wmh4k_8a02d5b3-ea9e-4f73-b2d7-527250090419/kube-rbac-proxy-metrics/0.log"
Jul 02 14:07:50.650878 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:50.650847    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cdb5968b9-kdlkr_b61b8094-e8b8-406b-89db-3c580535536d/networking-console-plugin/0.log"
Jul 02 14:07:52.026272 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.026241    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-69d9f679db-prr7j_0604d44c-2c9d-4260-8edb-64b85f9c321c/volume-data-source-validator/0.log"
Jul 02 14:07:52.049319 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.049289    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"]
Jul 02 14:07:52.049660 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.049648    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33d3477d-bffc-4436-8318-1c005caac047" containerName="copy"
Jul 02 14:07:52.049707 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.049663    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="33d3477d-bffc-4436-8318-1c005caac047" containerName="copy"
Jul 02 14:07:52.049707 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.049674    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f204ab2-93ae-427e-ac6d-d436baa331e9" containerName="kserve-container"
Jul 02 14:07:52.049707 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.049680    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f204ab2-93ae-427e-ac6d-d436baa331e9" containerName="kserve-container"
Jul 02 14:07:52.049707 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.049698    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33d3477d-bffc-4436-8318-1c005caac047" containerName="gather"
Jul 02 14:07:52.049707 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.049704    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="33d3477d-bffc-4436-8318-1c005caac047" containerName="gather"
Jul 02 14:07:52.049867 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.049712    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f204ab2-93ae-427e-ac6d-d436baa331e9" containerName="kube-rbac-proxy"
Jul 02 14:07:52.049867 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.049717    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f204ab2-93ae-427e-ac6d-d436baa331e9" containerName="kube-rbac-proxy"
Jul 02 14:07:52.049867 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.049771    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f204ab2-93ae-427e-ac6d-d436baa331e9" containerName="kserve-container"
Jul 02 14:07:52.049867 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.049777    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f204ab2-93ae-427e-ac6d-d436baa331e9" containerName="kube-rbac-proxy"
Jul 02 14:07:52.049867 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.049783    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="33d3477d-bffc-4436-8318-1c005caac047" containerName="gather"
Jul 02 14:07:52.049867 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.049789    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="33d3477d-bffc-4436-8318-1c005caac047" containerName="copy"
Jul 02 14:07:52.052764 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.052747    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.054460 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.054438    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5gcbj\"/\"openshift-service-ca.crt\""
Jul 02 14:07:52.054794 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.054777    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5gcbj\"/\"default-dockercfg-9lfdl\""
Jul 02 14:07:52.054891 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.054781    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5gcbj\"/\"kube-root-ca.crt\""
Jul 02 14:07:52.061956 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.061934    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"]
Jul 02 14:07:52.212098 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.212061    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl4cr\" (UniqueName: \"kubernetes.io/projected/dbf99d09-b048-4155-82de-22c73aa4506f-kube-api-access-zl4cr\") pod \"perf-node-gather-daemonset-b7snh\" (UID: \"dbf99d09-b048-4155-82de-22c73aa4506f\") " pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.212266 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.212108    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dbf99d09-b048-4155-82de-22c73aa4506f-podres\") pod \"perf-node-gather-daemonset-b7snh\" (UID: \"dbf99d09-b048-4155-82de-22c73aa4506f\") " pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.212266 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.212157    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dbf99d09-b048-4155-82de-22c73aa4506f-lib-modules\") pod \"perf-node-gather-daemonset-b7snh\" (UID: \"dbf99d09-b048-4155-82de-22c73aa4506f\") " pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.212266 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.212234    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbf99d09-b048-4155-82de-22c73aa4506f-sys\") pod \"perf-node-gather-daemonset-b7snh\" (UID: \"dbf99d09-b048-4155-82de-22c73aa4506f\") " pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.212266 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.212249    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dbf99d09-b048-4155-82de-22c73aa4506f-proc\") pod \"perf-node-gather-daemonset-b7snh\" (UID: \"dbf99d09-b048-4155-82de-22c73aa4506f\") " pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.313242 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.313145    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbf99d09-b048-4155-82de-22c73aa4506f-sys\") pod \"perf-node-gather-daemonset-b7snh\" (UID: \"dbf99d09-b048-4155-82de-22c73aa4506f\") " pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.313242 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.313196    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dbf99d09-b048-4155-82de-22c73aa4506f-proc\") pod \"perf-node-gather-daemonset-b7snh\" (UID: \"dbf99d09-b048-4155-82de-22c73aa4506f\") " pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.313455 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.313269    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dbf99d09-b048-4155-82de-22c73aa4506f-proc\") pod \"perf-node-gather-daemonset-b7snh\" (UID: \"dbf99d09-b048-4155-82de-22c73aa4506f\") " pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.313455 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.313278    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbf99d09-b048-4155-82de-22c73aa4506f-sys\") pod \"perf-node-gather-daemonset-b7snh\" (UID: \"dbf99d09-b048-4155-82de-22c73aa4506f\") " pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.313455 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.313299    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl4cr\" (UniqueName: \"kubernetes.io/projected/dbf99d09-b048-4155-82de-22c73aa4506f-kube-api-access-zl4cr\") pod \"perf-node-gather-daemonset-b7snh\" (UID: \"dbf99d09-b048-4155-82de-22c73aa4506f\") " pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.313455 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.313357    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dbf99d09-b048-4155-82de-22c73aa4506f-podres\") pod \"perf-node-gather-daemonset-b7snh\" (UID: \"dbf99d09-b048-4155-82de-22c73aa4506f\") " pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.313455 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.313431    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dbf99d09-b048-4155-82de-22c73aa4506f-lib-modules\") pod \"perf-node-gather-daemonset-b7snh\" (UID: \"dbf99d09-b048-4155-82de-22c73aa4506f\") " pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.313615 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.313516    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dbf99d09-b048-4155-82de-22c73aa4506f-podres\") pod \"perf-node-gather-daemonset-b7snh\" (UID: \"dbf99d09-b048-4155-82de-22c73aa4506f\") " pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.313615 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.313576    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dbf99d09-b048-4155-82de-22c73aa4506f-lib-modules\") pod \"perf-node-gather-daemonset-b7snh\" (UID: \"dbf99d09-b048-4155-82de-22c73aa4506f\") " pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.319871 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.319840    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl4cr\" (UniqueName: \"kubernetes.io/projected/dbf99d09-b048-4155-82de-22c73aa4506f-kube-api-access-zl4cr\") pod \"perf-node-gather-daemonset-b7snh\" (UID: \"dbf99d09-b048-4155-82de-22c73aa4506f\") " pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.362950 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.362911    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.488274 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.488248    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"]
Jul 02 14:07:52.490310 ip-10-0-131-238 kubenswrapper[2570]: W0702 14:07:52.490281    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddbf99d09_b048_4155_82de_22c73aa4506f.slice/crio-4f8e82abefe8dbc1c05f5912cd2ad0ff45e3072e83a9da308ab90b38142bbaa5 WatchSource:0}: Error finding container 4f8e82abefe8dbc1c05f5912cd2ad0ff45e3072e83a9da308ab90b38142bbaa5: Status 404 returned error can't find the container with id 4f8e82abefe8dbc1c05f5912cd2ad0ff45e3072e83a9da308ab90b38142bbaa5
Jul 02 14:07:52.681698 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.681597    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh" event={"ID":"dbf99d09-b048-4155-82de-22c73aa4506f","Type":"ContainerStarted","Data":"6f4d2cb2cf25e8e9e28f6a90ff464f7ac399113076c282e76e40886d9bf611b3"}
Jul 02 14:07:52.681698 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.681650    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh" event={"ID":"dbf99d09-b048-4155-82de-22c73aa4506f","Type":"ContainerStarted","Data":"4f8e82abefe8dbc1c05f5912cd2ad0ff45e3072e83a9da308ab90b38142bbaa5"}
Jul 02 14:07:52.681698 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.681685    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:07:52.695485 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.695443    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh" podStartSLOduration=0.695428201 podStartE2EDuration="695.428201ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:07:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:07:52.69370808 +0000 UTC m=+3224.316500999" watchObservedRunningTime="2026-07-02 14:07:52.695428201 +0000 UTC m=+3224.318221107"
Jul 02 14:07:52.916673 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.916639    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h66tv_0971a04b-36bc-46ca-9bce-762b11f50d31/dns/0.log"
Jul 02 14:07:52.932826 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:52.932738    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h66tv_0971a04b-36bc-46ca-9bce-762b11f50d31/kube-rbac-proxy/0.log"
Jul 02 14:07:53.008173 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:53.008138    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wfmns_fb05ec42-ba16-4d8b-82e1-1e689b75392d/dns-node-resolver/0.log"
Jul 02 14:07:53.513904 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:53.513870    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-q9xvb_cc015ede-26b1-49e2-9812-626d373e658f/node-ca/0.log"
Jul 02 14:07:54.639900 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:54.639872    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dfhv6_6b13232c-9bcc-423b-a85f-1ef580a1d477/serve-healthcheck-canary/0.log"
Jul 02 14:07:54.984427 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:54.984397    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5f67b55555-847dx_273bcb09-ea54-4145-af21-7392e86cdb9b/insights-operator/0.log"
Jul 02 14:07:54.984950 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:54.984919    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5f67b55555-847dx_273bcb09-ea54-4145-af21-7392e86cdb9b/insights-operator/1.log"
Jul 02 14:07:55.106894 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:55.106857    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v8fdm_30373957-8786-4661-8e15-0a26ec06fd66/kube-rbac-proxy/0.log"
Jul 02 14:07:55.123325 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:55.123298    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v8fdm_30373957-8786-4661-8e15-0a26ec06fd66/exporter/0.log"
Jul 02 14:07:55.142512 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:55.142491    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v8fdm_30373957-8786-4661-8e15-0a26ec06fd66/extractor/0.log"
Jul 02 14:07:56.688443 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:56.688410    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9z5jl_ae596d11-edc3-4966-abdb-f76a7b0f9c94/konnectivity-agent/0.log" limit=1024
Jul 02 14:07:57.196513 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:57.196477    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-fd65d7d6-gk29w_c99e727e-7b07-4e77-8452-ce73f33cd3f6/server/0.log"
Jul 02 14:07:57.278056 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:57.278016    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-ddtp6_94c22db7-55c6-4708-aba0-c2c1c9cc9576/manager/0.log"
Jul 02 14:07:57.305622 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:57.305590    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-tp4l8_e68a0a1b-e0c6-412a-a601-e96804858c87/s3-init/0.log"
Jul 02 14:07:57.318088 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:57.318065    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-tp4l8_e68a0a1b-e0c6-412a-a601-e96804858c87/download-hf-model/0.log"
Jul 02 14:07:57.339648 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:57.339623    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-nnnjb_af9df460-cc0b-4046-b579-e1332211dd14/seaweedfs/0.log"
Jul 02 14:07:58.695821 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:07:58.695795    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5gcbj/perf-node-gather-daemonset-b7snh"
Jul 02 14:08:00.961726 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:00.961698    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-5dkvq_a9972a0a-39f3-4dc1-b478-cbaa91d3a3b0/migrator/0.log"
Jul 02 14:08:00.982226 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:00.982199    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-5dkvq_a9972a0a-39f3-4dc1-b478-cbaa91d3a3b0/graceful-termination/0.log"
Jul 02 14:08:02.582747 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:02.582720    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwphv_11cd0142-78dd-4651-b2af-2c26f1ce70e4/kube-multus-additional-cni-plugins/0.log"
Jul 02 14:08:02.600086 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:02.600059    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwphv_11cd0142-78dd-4651-b2af-2c26f1ce70e4/egress-router-binary-copy/0.log"
Jul 02 14:08:02.618021 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:02.617989    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwphv_11cd0142-78dd-4651-b2af-2c26f1ce70e4/cni-plugins/0.log"
Jul 02 14:08:02.635257 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:02.635234    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwphv_11cd0142-78dd-4651-b2af-2c26f1ce70e4/bond-cni-plugin/0.log"
Jul 02 14:08:02.653509 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:02.653487    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwphv_11cd0142-78dd-4651-b2af-2c26f1ce70e4/routeoverride-cni/0.log"
Jul 02 14:08:02.671116 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:02.671095    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwphv_11cd0142-78dd-4651-b2af-2c26f1ce70e4/whereabouts-cni-bincopy/0.log"
Jul 02 14:08:02.689474 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:02.689452    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwphv_11cd0142-78dd-4651-b2af-2c26f1ce70e4/whereabouts-cni/0.log"
Jul 02 14:08:02.719570 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:02.719545    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-g4qzs_9454c557-e08b-4fbd-92fe-a2cffddf034b/kube-multus/0.log"
Jul 02 14:08:02.800796 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:02.800768    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6nhcs_aa2c8da0-b138-46ef-b9eb-10f6922697be/network-metrics-daemon/0.log"
Jul 02 14:08:02.817791 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:02.817759    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6nhcs_aa2c8da0-b138-46ef-b9eb-10f6922697be/kube-rbac-proxy/0.log"
Jul 02 14:08:03.661337 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:03.661259    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-controller/0.log"
Jul 02 14:08:03.675620 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:03.675601    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/1.log"
Jul 02 14:08:03.675812 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:03.675796    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovn-acl-logging/0.log"
Jul 02 14:08:03.694673 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:03.694651    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/kube-rbac-proxy-node/0.log"
Jul 02 14:08:03.711598 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:03.711554    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/kube-rbac-proxy-ovn-metrics/0.log"
Jul 02 14:08:03.728745 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:03.728726    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/northd/0.log"
Jul 02 14:08:03.745524 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:03.745500    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/nbdb/0.log"
Jul 02 14:08:03.765659 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:03.765636    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/sbdb/0.log"
Jul 02 14:08:03.807804 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:03.807776    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djnhg_8cce14c3-bcec-404c-9873-9825555f2fc4/ovnkube-controller/0.log"
Jul 02 14:08:05.057168 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:05.057138    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-56499946bc-twcbv_6b7d6fd3-387f-431c-88d4-43148130bac4/check-endpoints/0.log"
Jul 02 14:08:05.100275 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:05.100243    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-kv69m_c41c0fa4-04c6-4e1b-9af2-0635451f3f26/network-check-target-container/0.log"
Jul 02 14:08:05.957625 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:05.957596    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-f4rtw_877fea60-1134-4e22-8db9-67000f4a4692/iptables-alerter/0.log"
Jul 02 14:08:06.620162 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:06.620133    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-tt9pg_9325a868-a924-4308-8533-25b57f13535a/tuned/0.log"
Jul 02 14:08:08.207108 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:08.207077    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-9ddc5ff76-5qjjn_af4235e1-a082-492c-aebb-7ef694d20028/cluster-samples-operator/0.log"
Jul 02 14:08:08.220715 ip-10-0-131-238 kubenswrapper[2570]: I0702 14:08:08.220690    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-9ddc5ff76-5qjjn_af4235e1-a082-492c-aebb-7ef694d20028/cluster-samples-operator-watch/0.log"