Jul 31 23:44:32.018268 ip-10-0-139-141 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 31 23:44:32.018277 ip-10-0-139-141 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 31 23:44:32.018284 ip-10-0-139-141 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 31 23:44:32.018505 ip-10-0-139-141 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 31 23:44:42.190006 ip-10-0-139-141 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 31 23:44:42.190028 ip-10-0-139-141 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot fea863aa2f5f4aff8fc3e4fc5ed8a789 --
Jul 31 23:46:16.689275 ip-10-0-139-141 systemd[1]: Starting Kubernetes Kubelet...
Jul 31 23:46:17.140618 ip-10-0-139-141 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 31 23:46:17.140618 ip-10-0-139-141 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 31 23:46:17.140618 ip-10-0-139-141 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 31 23:46:17.140618 ip-10-0-139-141 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 31 23:46:17.140618 ip-10-0-139-141 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 31 23:46:17.143624 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.143497    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 31 23:46:17.150010 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.149978    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 23:46:17.150010 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150003    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 23:46:17.150010 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150009    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 23:46:17.150010 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150012    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 23:46:17.150010 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150019    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150024    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150028    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150032    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150035    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150039    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150043    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150046    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150050    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150053    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150056    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150061    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150067    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150070    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150073    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150075    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150078    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150081    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150084    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 23:46:17.150229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150087    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150090    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150093    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150095    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150098    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150101    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150109    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150111    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150114    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150117    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150120    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150122    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150125    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150128    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150130    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150134    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150137    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150140    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150145    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150148    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 23:46:17.150686 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150151    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150154    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150156    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150160    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150162    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150165    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150168    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150170    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150173    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150176    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150181    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150184    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150187    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150190    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150192    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150195    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150198    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150200    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150203    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150206    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 23:46:17.151222 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150209    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150211    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150214    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150222    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150226    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150229    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150232    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150236    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150239    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150242    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150246    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150248    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150251    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150254    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150258    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150263    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150266    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150268    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150271    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150274    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 23:46:17.151704 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150277    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150280    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150283    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150285    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150289    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150292    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150294    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150300    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150302    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150305    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150308    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150311    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150313    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150316    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150318    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.150321    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151569    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151576    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151580    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151584    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 23:46:17.152341 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151587    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151590    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151593    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151596    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151599    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151602    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151614    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151617    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151620    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151623    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151627    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151629    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151632    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151635    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151640    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151644    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151648    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151651    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151657    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151660    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 23:46:17.152915 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151663    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151666    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151669    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151672    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151675    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151678    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151681    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151683    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151686    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151689    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151692    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151697    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151700    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151702    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151705    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151708    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151711    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151714    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151717    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151733    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 23:46:17.153462 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151736    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151739    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151741    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151744    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151749    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151752    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151755    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151757    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151760    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151763    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151766    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151772    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151776    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151779    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151782    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151786    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151791    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151794    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151797    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 23:46:17.153988 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151802    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151805    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151808    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151812    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151814    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151862    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151880    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151886    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.151935    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152097    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152104    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152107    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152111    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152114    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152117    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152120    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152123    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152126    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152129    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152132    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 23:46:17.154491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152134    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152139    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152141    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152144    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152150    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152152    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152155    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152158    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152161    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152164    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152166    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152169    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152172    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152175    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152177    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152180    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152685    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152691    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152694    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152696    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 23:46:17.155008 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152699    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152703    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152706    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152709    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152712    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152715    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152718    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152721    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152724    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152727    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152729    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152732    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152735    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152737    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152741    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152743    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152746    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152748    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152751    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 23:46:17.155490 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152755    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152759    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152762    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152765    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152768    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152771    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152774    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152777    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152779    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152782    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152785    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152788    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152791    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152793    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152796    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152800    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152802    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152805    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152808    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152811    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152813    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 23:46:17.155940 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152816    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152818    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152821    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152823    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152826    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152828    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152831    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152833    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152836    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152838    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152841    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152844    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152846    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152849    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152852    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152854    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152857    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152859    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152862    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 23:46:17.156491 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152864    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152867    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152869    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152872    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152874    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152877    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152881    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152883    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152886    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152891    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152895    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152897    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152900    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152903    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152905    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152908    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152910    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152913    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152916    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 23:46:17.156987 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152924    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152927    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152930    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152933    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152935    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152938    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152940    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152943    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152945    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152948    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152950    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152974    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152979    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152981    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152984    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152988    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.152992    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153066    2581 flags.go:64] FLAG: --address="0.0.0.0"
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153074    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153081    2581 flags.go:64] FLAG: --anonymous-auth="true"
Jul 31 23:46:17.157463 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153086    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153093    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153096    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153101    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153105    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153109    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153112    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153115    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153119    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153122    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153125    2581 flags.go:64] FLAG: --cgroup-root=""
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153128    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153132    2581 flags.go:64] FLAG: --client-ca-file=""
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153152    2581 flags.go:64] FLAG: --cloud-provider="external"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153156    2581 flags.go:64] FLAG: --cluster-dns="[]"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153163    2581 flags.go:64] FLAG: --cluster-domain=""
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153166    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153169    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153172    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153176    2581 flags.go:64] FLAG: --container-log-max-files="5"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153184    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153187    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153195    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 31 23:46:17.157994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153199    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153202    2581 flags.go:64] FLAG: --contention-profiling="false"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153205    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153209    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153212    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153215    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153219    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153222    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153225    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153228    2581 flags.go:64] FLAG: --enable-load-reader="false"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153231    2581 flags.go:64] FLAG: --enable-server="true"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153234    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153241    2581 flags.go:64] FLAG: --event-burst="100"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153244    2581 flags.go:64] FLAG: --event-qps="50"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153246    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153249    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153252    2581 flags.go:64] FLAG: --eviction-hard=""
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153256    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153259    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153262    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153266    2581 flags.go:64] FLAG: --eviction-soft=""
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153269    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153272    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153280    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153283    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 31 23:46:17.158566 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153286    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153289    2581 flags.go:64] FLAG: --fail-swap-on="true"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153292    2581 flags.go:64] FLAG: --feature-gates=""
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153296    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153299    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153302    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153307    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153310    2581 flags.go:64] FLAG: --healthz-port="10248"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153313    2581 flags.go:64] FLAG: --help="false"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153316    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153319    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153322    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153325    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153328    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153332    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153336    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153339    2581 flags.go:64] FLAG: --image-service-endpoint=""
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153342    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153345    2581 flags.go:64] FLAG: --kube-api-burst="100"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153348    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153352    2581 flags.go:64] FLAG: --kube-api-qps="50"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153355    2581 flags.go:64] FLAG: --kube-reserved=""
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153358    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153361    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 31 23:46:17.159285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153364    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153367    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153370    2581 flags.go:64] FLAG: --lock-file=""
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153373    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153376    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153379    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153384    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153393    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153396    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153399    2581 flags.go:64] FLAG: --logging-format="text"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153402    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153405    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153408    2581 flags.go:64] FLAG: --manifest-url=""
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153411    2581 flags.go:64] FLAG: --manifest-url-header=""
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153417    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153420    2581 flags.go:64] FLAG: --max-open-files="1000000"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153424    2581 flags.go:64] FLAG: --max-pods="110"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153427    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153431    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153434    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153436    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153439    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153442    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153447    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153454    2581 flags.go:64] FLAG: --node-status-max-images="50"
Jul 31 23:46:17.159884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153458    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153461    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153464    2581 flags.go:64] FLAG: --pod-cidr=""
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153467    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153473    2581 flags.go:64] FLAG: --pod-manifest-path=""
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153476    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153479    2581 flags.go:64] FLAG: --pods-per-core="0"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153482    2581 flags.go:64] FLAG: --port="10250"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153485    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153488    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0417017ec34594a43"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153491    2581 flags.go:64] FLAG: --qos-reserved=""
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153494    2581 flags.go:64] FLAG: --read-only-port="10255"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153497    2581 flags.go:64] FLAG: --register-node="true"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153500    2581 flags.go:64] FLAG: --register-with-taints=""
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153504    2581 flags.go:64] FLAG: --registry-burst="10"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153513    2581 flags.go:64] FLAG: --registry-qps="5"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153516    2581 flags.go:64] FLAG: --reserved-cpus=""
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153518    2581 flags.go:64] FLAG: --reserved-memory=""
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153522    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153525    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153528    2581 flags.go:64] FLAG: --rotate-certificates="false"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153532    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153535    2581 flags.go:64] FLAG: --runonce="false"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153538    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153541    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 31 23:46:17.160521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153544    2581 flags.go:64] FLAG: --seccomp-default="false"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153547    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153550    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153554    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153557    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153560    2581 flags.go:64] FLAG: --storage-driver-password="root"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153564    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153567    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153570    2581 flags.go:64] FLAG: --storage-driver-user="root"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153573    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153576    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153579    2581 flags.go:64] FLAG: --system-cgroups=""
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153582    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153588    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153590    2581 flags.go:64] FLAG: --tls-cert-file=""
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153593    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153600    2581 flags.go:64] FLAG: --tls-min-version=""
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153603    2581 flags.go:64] FLAG: --tls-private-key-file=""
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153607    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153609    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153612    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153616    2581 flags.go:64] FLAG: --v="2"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153620    2581 flags.go:64] FLAG: --version="false"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153629    2581 flags.go:64] FLAG: --vmodule=""
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153633    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 31 23:46:17.161168 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.153637    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153777    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153781    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153784    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153787    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153791    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153795    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153798    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153801    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153804    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153807    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153810    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153812    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153816    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153819    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153822    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153824    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153827    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153830    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 23:46:17.161793 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153833    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153836    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153839    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153841    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153844    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153847    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153849    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153852    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153854    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153857    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153860    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153864    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153873    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153876    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153879    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153881    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153884    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153887    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153889    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 23:46:17.162283 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153892    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153894    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153897    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153899    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153902    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153904    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153907    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153912    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153915    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153918    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153920    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153923    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153925    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153928    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153931    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153934    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153936    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153939    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153941    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153944    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 23:46:17.162747 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153946    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153949    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153951    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153970    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153973    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153976    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153985    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153988    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153991    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153993    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153996    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.153999    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154002    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154005    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154007    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154010    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154012    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154016    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154019    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154024    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 23:46:17.163259 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154027    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154030    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154032    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154035    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154038    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154040    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154043    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154045    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154050    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154053    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154055    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154058    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154061    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154063    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154066    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154068    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154071    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154073    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154076    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 23:46:17.163769 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154084    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 23:46:17.164229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154087    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 23:46:17.164229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.154089    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 23:46:17.164229 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.155063    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 31 23:46:17.164229 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.162869    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 31 23:46:17.164229 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.162888    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 31 23:46:17.164229 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.163912    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 31 23:46:17.164229 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.163923    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 31 23:46:17.164229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164005    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 23:46:17.164229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164012    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 23:46:17.164229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164015    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 23:46:17.164229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164019    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 23:46:17.164229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164022    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 23:46:17.164229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164025    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 23:46:17.164229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164028    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 23:46:17.164229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164031    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164034    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164039    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164043    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164046    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164050    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164053    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164056    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164059    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164062    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164065    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164068    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164071    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164074    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164078    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164081    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164083    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164087    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164090    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 23:46:17.164600 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164093    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164095    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164098    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164102    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164105    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164108    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164111    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164114    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164117    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164120    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164122    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164125    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164127    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164130    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164133    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164136    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164138    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164141    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164146    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164149    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 23:46:17.165073 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164152    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164155    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164158    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164161    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164163    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164166    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164168    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164171    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164174    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164177    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164179    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164182    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164185    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164188    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164191    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164195    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164197    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164200    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164203    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164206    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 23:46:17.165560 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164208    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164211    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164213    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164216    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164219    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164221    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164224    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164226    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164229    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164232    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164235    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164238    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164240    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164243    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164246    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164248    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164251    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164253    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164256    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 23:46:17.166061 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164258    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 23:46:17.166515 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164263    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 23:46:17.166515 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164266    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 23:46:17.166515 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164270    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 23:46:17.166515 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164273    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 23:46:17.166515 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164276    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 23:46:17.166515 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164279    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 23:46:17.166515 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164282    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 23:46:17.166515 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164286    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 23:46:17.166515 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164289    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 23:46:17.166515 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164291    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 23:46:17.166515 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164294    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 23:46:17.166515 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164297    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 23:46:17.166515 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164299    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 23:46:17.166515 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.164305    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 31 23:46:17.166515 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164425    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164429    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164432    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164435    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164438    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164441    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164444    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164446    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164449    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164452    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164454    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164457    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164460    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164462    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164465    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164467    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164470    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164472    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164475    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164477    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 23:46:17.166935 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164480    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164482    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164486    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164496    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164499    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164502    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164505    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164507    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164510    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164512    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164515    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164518    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164520    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164523    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164525    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164528    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164531    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164533    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164536    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 23:46:17.167443 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164539    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164542    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164544    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164547    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164549    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164552    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164554    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164557    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164560    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164562    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164565    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164567    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164570    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164573    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164576    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164579    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164581    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164588    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164591    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164594    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164596    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 23:46:17.167900 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164599    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164603    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164607    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164610    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164613    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164616    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164619    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164622    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164625    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164628    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164632    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164635    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164638    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164641    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164644    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164647    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164650    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164653    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164656    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 23:46:17.168422 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164659    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164661    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164664    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164666    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164669    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164672    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164674    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164677    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164680    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164682    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164685    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164688    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164691    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164693    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164696    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164699    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164701    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164704    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164706    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 23:46:17.168889 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:17.164709    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 23:46:17.169421 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.164714    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 31 23:46:17.169421 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.164853    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 31 23:46:17.169421 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.167417    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 31 23:46:17.169421 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.169069    2581 server.go:1019] "Starting client certificate rotation"
Jul 31 23:46:17.169421 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.169178    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 31 23:46:17.169421 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.169212    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 31 23:46:17.172105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.172089    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 31 23:46:17.173301 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.173282    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 31 23:46:17.190342 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.190316    2581 log.go:25] "Validated CRI v1 runtime API"
Jul 31 23:46:17.193002 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.192983    2581 log.go:25] "Validated CRI v1 image API"
Jul 31 23:46:17.193819 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.193794    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 31 23:46:17.196646 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.196621    2581 fs.go:135] Filesystem UUIDs: map[2b9d09f8-f202-466d-a6d9-9aa45cf23d43:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f16a3a09-460b-4ba9-a71d-101501110866:/dev/nvme0n1p4]
Jul 31 23:46:17.196704 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.196646    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 31 23:46:17.203749 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.203625    2581 manager.go:216] Machine: {Timestamp:2026-07-31 23:46:17.202250224 +0000 UTC m=+0.381706958 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3109076 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bdb3ae4144020ee3ce3803a581778 SystemUUID:ec2bdb3a-e414-4020-ee3c-e3803a581778 BootID:fea863aa-2f5f-4aff-8fc3-e4fc5ed8a789 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c0:65:54:5a:fd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c0:65:54:5a:fd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9a:04:a0:ef:df:e5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 31 23:46:17.203749 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.203733    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 31 23:46:17.203909 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.203825    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 31 23:46:17.204083 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.204065    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 31 23:46:17.205733 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.205710    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 31 23:46:17.205870 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.205737    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-141.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 31 23:46:17.206597 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.206586    2581 topology_manager.go:138] "Creating topology manager with none policy"
Jul 31 23:46:17.206597 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.206598    2581 container_manager_linux.go:309] "Creating device plugin manager"
Jul 31 23:46:17.206681 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.206606    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 31 23:46:17.207478 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.207468    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 31 23:46:17.208669 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.208660    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 31 23:46:17.213898 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.213888    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 31 23:46:17.214042 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.214032    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 31 23:46:17.214082 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.214064    2581 kubelet.go:486] "Attempting to sync node with API server"
Jul 31 23:46:17.214082 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.214079    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 31 23:46:17.214133 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.214092    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 31 23:46:17.214133 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.214102    2581 kubelet.go:398] "Adding apiserver pod source"
Jul 31 23:46:17.214133 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.214110    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 31 23:46:17.214477 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.214465    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 31 23:46:17.214522 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.214482    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 31 23:46:17.216860 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.216844    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 31 23:46:17.218771 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.218755    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 31 23:46:17.218828 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.218798    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 31 23:46:17.220065 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.220038    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sq86v"
Jul 31 23:46:17.220195 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.220181    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 31 23:46:17.220235 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.220199    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 31 23:46:17.220235 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.220205    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 31 23:46:17.220235 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.220210    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 31 23:46:17.220235 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.220219    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 31 23:46:17.220235 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.220225    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 31 23:46:17.220235 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.220232    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 31 23:46:17.220391 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.220239    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 31 23:46:17.220391 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.220245    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 31 23:46:17.220391 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.220251    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 31 23:46:17.220391 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.220259    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 31 23:46:17.220391 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.220265    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 31 23:46:17.221931 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.221921    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 31 23:46:17.221931 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.221931    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 31 23:46:17.224577 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.224557    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-141.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 31 23:46:17.224791 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.224763    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 31 23:46:17.225482 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.225467    2581 server.go:1268] "Started kubelet"
Jul 31 23:46:17.225634 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.225590    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 31 23:46:17.225712 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.225628    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 31 23:46:17.225712 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.225656    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 31 23:46:17.226431 ip-10-0-139-141 systemd[1]: Started Kubernetes Kubelet.
Jul 31 23:46:17.228618 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.228602    2581 server.go:310] "Adding debug handlers to kubelet server"
Jul 31 23:46:17.229457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.229438    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sq86v"
Jul 31 23:46:17.230276 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.230259    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 31 23:46:17.231362 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.231337    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 31 23:46:17.231858 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.231835    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 31 23:46:17.232484 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.231368    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-141.ec2.internal.18c7849ab236789a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-141.ec2.internal,UID:ip-10-0-139-141.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-141.ec2.internal,},FirstTimestamp:2026-07-31 23:46:17.22543529 +0000 UTC m=+0.404892027,LastTimestamp:2026-07-31 23:46:17.22543529 +0000 UTC m=+0.404892027,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-141.ec2.internal,}"
Jul 31 23:46:17.238571 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.238547    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 31 23:46:17.238656 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.238590    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 31 23:46:17.238711 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.238664    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-141.ec2.internal\" not found"
Jul 31 23:46:17.239329 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.239307    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 31 23:46:17.239434 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.239351    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 31 23:46:17.239434 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.239365    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 31 23:46:17.239552 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.239494    2581 reconstruct.go:97] "Volume reconstruction finished"
Jul 31 23:46:17.239552 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.239505    2581 reconciler.go:29] "Reconciler: start to sync state"
Jul 31 23:46:17.239652 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.239560    2581 factory.go:55] Registering systemd factory
Jul 31 23:46:17.239652 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.239577    2581 factory.go:223] Registration of the systemd container factory successfully
Jul 31 23:46:17.240744 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.240034    2581 factory.go:153] Registering CRI-O factory
Jul 31 23:46:17.240744 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.240055    2581 factory.go:223] Registration of the crio container factory successfully
Jul 31 23:46:17.240744 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.240113    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 31 23:46:17.240744 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.240160    2581 factory.go:103] Registering Raw factory
Jul 31 23:46:17.240744 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.240177    2581 manager.go:1196] Started watching for new ooms in manager
Jul 31 23:46:17.242206 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.242145    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 31 23:46:17.242294 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.242232    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 31 23:46:17.242360 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.242351    2581 manager.go:349] Starting recovery of all containers
Jul 31 23:46:17.253364 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.253340    2581 manager.go:354] Recovery completed
Jul 31 23:46:17.257681 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.257668    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 23:46:17.259512 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.259496    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 23:46:17.259577 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.259525    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 23:46:17.259577 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.259535    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasSufficientPID"
Jul 31 23:46:17.260077 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.260061    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 31 23:46:17.260077 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.260075    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 31 23:46:17.260208 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.260084    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 23:46:17.260208 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.260094    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 31 23:46:17.260808 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.260796    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 23:46:17.260846 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.260821    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 23:46:17.260846 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.260831    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasSufficientPID"
Jul 31 23:46:17.262209 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.262195    2581 policy_none.go:49] "None policy: Start"
Jul 31 23:46:17.262273 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.262213    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 31 23:46:17.262273 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.262223    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 31 23:46:17.264180 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.264168    2581 policy_none.go:47] "Start"
Jul 31 23:46:17.303729 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.303540    2581 manager.go:337] "Starting Device Plugin manager"
Jul 31 23:46:17.320826 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.303763    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 31 23:46:17.320826 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.303780    2581 server.go:87] "Starting device plugin registration server"
Jul 31 23:46:17.320826 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.304102    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 31 23:46:17.320826 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.304116    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 31 23:46:17.320826 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.304213    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 31 23:46:17.320826 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.304298    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 31 23:46:17.320826 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.304308    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 31 23:46:17.320826 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.304934    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 31 23:46:17.320826 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.304993    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-141.ec2.internal\" not found"
Jul 31 23:46:17.380835 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.380785    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 31 23:46:17.381905 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.381891    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 31 23:46:17.381994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.381930    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 31 23:46:17.381994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.381967    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 31 23:46:17.382068 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.382001    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 31 23:46:17.384113 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.384088    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 23:46:17.404911 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.404843    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 23:46:17.405739 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.405722    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 23:46:17.405852 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.405758    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 23:46:17.405852 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.405772    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasSufficientPID"
Jul 31 23:46:17.405852 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.405801    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.416075 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.416054    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.416180 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.416082    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-141.ec2.internal\": node \"ip-10-0-139-141.ec2.internal\" not found"
Jul 31 23:46:17.435455 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.435427    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-141.ec2.internal\" not found"
Jul 31 23:46:17.482150 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.482120    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-141.ec2.internal"]
Jul 31 23:46:17.482307 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.482207    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 23:46:17.483716 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.483698    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 23:46:17.483781 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.483729    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 23:46:17.483781 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.483739    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasSufficientPID"
Jul 31 23:46:17.486104 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.486091    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 23:46:17.486196 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.486181    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.486251 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.486211    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 23:46:17.486801 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.486785    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 23:46:17.486860 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.486813    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 23:46:17.486860 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.486825    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasSufficientPID"
Jul 31 23:46:17.486930 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.486789    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 23:46:17.486930 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.486884    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 23:46:17.486930 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.486895    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasSufficientPID"
Jul 31 23:46:17.489162 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.489142    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.489269 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.489171    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 23:46:17.489860 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.489844    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 23:46:17.489936 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.489871    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 23:46:17.489936 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.489883    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeHasSufficientPID"
Jul 31 23:46:17.520781 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.520754    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-141.ec2.internal\" not found" node="ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.526560 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.526537    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-141.ec2.internal\" not found" node="ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.535675 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.535651    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-141.ec2.internal\" not found"
Jul 31 23:46:17.542042 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.542015    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8a14086636be88f16b8a69fdf15ede03-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal\" (UID: \"8a14086636be88f16b8a69fdf15ede03\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.542126 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.542046    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a14086636be88f16b8a69fdf15ede03-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal\" (UID: \"8a14086636be88f16b8a69fdf15ede03\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.542126 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.542064    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/109099d262acad3a59990c8222269413-config\") pod \"kube-apiserver-proxy-ip-10-0-139-141.ec2.internal\" (UID: \"109099d262acad3a59990c8222269413\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.636375 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.636342    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-141.ec2.internal\" not found"
Jul 31 23:46:17.643207 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.643160    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8a14086636be88f16b8a69fdf15ede03-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal\" (UID: \"8a14086636be88f16b8a69fdf15ede03\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.643289 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.643214    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a14086636be88f16b8a69fdf15ede03-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal\" (UID: \"8a14086636be88f16b8a69fdf15ede03\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.643289 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.643234    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/109099d262acad3a59990c8222269413-config\") pod \"kube-apiserver-proxy-ip-10-0-139-141.ec2.internal\" (UID: \"109099d262acad3a59990c8222269413\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.643374 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.643284    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a14086636be88f16b8a69fdf15ede03-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal\" (UID: \"8a14086636be88f16b8a69fdf15ede03\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.643374 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.643291    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8a14086636be88f16b8a69fdf15ede03-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal\" (UID: \"8a14086636be88f16b8a69fdf15ede03\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.643374 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.643329    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/109099d262acad3a59990c8222269413-config\") pod \"kube-apiserver-proxy-ip-10-0-139-141.ec2.internal\" (UID: \"109099d262acad3a59990c8222269413\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.738067 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.737949    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-141.ec2.internal\" not found"
Jul 31 23:46:17.829506 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.829457    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.829628 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:17.829458    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal"
Jul 31 23:46:17.839054 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.839031    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-141.ec2.internal\" not found"
Jul 31 23:46:17.943307 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:17.943262    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-141.ec2.internal\" not found"
Jul 31 23:46:18.043833 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:18.043745    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-141.ec2.internal\" not found"
Jul 31 23:46:18.145906 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:18.145875    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-141.ec2.internal\" not found"
Jul 31 23:46:18.168881 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:18.168845    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 31 23:46:18.169063 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:18.169034    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 31 23:46:18.236535 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:18.236477    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-30 23:41:18 +0000 UTC" deadline="2028-02-26 07:20:16.738543182 +0000 UTC"
Jul 31 23:46:18.236535 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:18.236526    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13783h33m58.502019651s"
Jul 31 23:46:18.246009 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:18.245981    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-141.ec2.internal\" not found"
Jul 31 23:46:18.256300 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:18.256279    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 23:46:18.346847 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:18.346769    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-141.ec2.internal\" not found"
Jul 31 23:46:18.392100 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:18.392059    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a14086636be88f16b8a69fdf15ede03.slice/crio-3a09ef6fa054323cc92335f7cd49ea0ee62c66a53001b02709674b5f73fb441f WatchSource:0}: Error finding container 3a09ef6fa054323cc92335f7cd49ea0ee62c66a53001b02709674b5f73fb441f: Status 404 returned error can't find the container with id 3a09ef6fa054323cc92335f7cd49ea0ee62c66a53001b02709674b5f73fb441f
Jul 31 23:46:18.400407 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:18.400387    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 23:46:18.406892 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:18.406867    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod109099d262acad3a59990c8222269413.slice/crio-2b809fb7d8e3da0b22e467d7897361d49c4ad2eec5af70efbb8cff486d665806 WatchSource:0}: Error finding container 2b809fb7d8e3da0b22e467d7897361d49c4ad2eec5af70efbb8cff486d665806: Status 404 returned error can't find the container with id 2b809fb7d8e3da0b22e467d7897361d49c4ad2eec5af70efbb8cff486d665806
Jul 31 23:46:18.447798 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:18.447762    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-141.ec2.internal\" not found"
Jul 31 23:46:18.548915 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:18.548885    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-141.ec2.internal\" not found"
Jul 31 23:46:18.555249 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:18.555231    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 23:46:18.645265 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:18.645152    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal"
Jul 31 23:46:18.656924 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:18.656889    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 31 23:46:18.657901 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:18.657882    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-141.ec2.internal"
Jul 31 23:46:18.667773 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:18.667744    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 31 23:46:18.732292 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:18.732259    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 23:46:19.215741 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.215707    2581 apiserver.go:52] "Watching apiserver"
Jul 31 23:46:19.225439 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.225410    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 31 23:46:19.225899 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.225866    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-141.ec2.internal","openshift-cluster-node-tuning-operator/tuned-dpmjk","openshift-dns/node-resolver-twdx5","openshift-image-registry/node-ca-d25h7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal","openshift-multus/multus-additional-cni-plugins-q6bv6","openshift-multus/multus-hftd5","openshift-network-diagnostics/network-check-target-frb4r","kube-system/global-pull-secret-syncer-4k4dl","kube-system/konnectivity-agent-zqrmw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf","openshift-multus/network-metrics-daemon-tn7kj","openshift-network-operator/iptables-alerter-c8qbc","openshift-ovn-kubernetes/ovnkube-node-5sqvw"]
Jul 31 23:46:19.228687 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.228659    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.231743 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.231714    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 31 23:46:19.232085 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.232066    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 31 23:46:19.232188 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.232110    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 31 23:46:19.232188 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.232110    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4g4ll\""
Jul 31 23:46:19.232299 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.232210    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 31 23:46:19.232672 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.232650    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.235112 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.235087    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-twdx5"
Jul 31 23:46:19.235669 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.235648    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 31 23:46:19.236148 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.236128    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 31 23:46:19.237165 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.237144    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-s2dww\""
Jul 31 23:46:19.237856 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.237832    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 31 23:46:19.238099 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.238079    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 31 23:46:19.239018 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.238792    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wbb5j\""
Jul 31 23:46:19.239105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.239025    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 31 23:46:19.239380 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.239359    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 31 23:46:19.241562 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.241209    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d25h7"
Jul 31 23:46:19.243745 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.243719    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cbzwg\""
Jul 31 23:46:19.243878 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.243711    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 31 23:46:19.243878 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.243838    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 31 23:46:19.244094 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.244076    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 31 23:46:19.244676 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.244653    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.247439 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.247416    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.250395 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.250376    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 31 23:46:19.250502 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.250413    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xxtcd\""
Jul 31 23:46:19.250502 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.250428    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 31 23:46:19.250728 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.250706    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:19.250869 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:19.250839    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:19.251778 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.251758    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dczv6\""
Jul 31 23:46:19.251778 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.251777    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 31 23:46:19.251999 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.251958    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 31 23:46:19.252082 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.252027    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 31 23:46:19.252214 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.252197    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 31 23:46:19.252276 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.252264    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 31 23:46:19.252276 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.252265    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 31 23:46:19.253330 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253302    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-var-lib-kubelet\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.253428 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253342    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-sys-fs\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.253488 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253432    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c8qbc"
Jul 31 23:46:19.253488 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253436    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzfcq\" (UniqueName: \"kubernetes.io/projected/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-kube-api-access-bzfcq\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.253488 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253467    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f85fb808-2a50-499e-a3a8-3aa1c33f69a2-hosts-file\") pod \"node-resolver-twdx5\" (UID: \"f85fb808-2a50-499e-a3a8-3aa1c33f69a2\") " pod="openshift-dns/node-resolver-twdx5"
Jul 31 23:46:19.253696 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253490    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5dd3564c-d0e5-444f-9417-4d23ea2951e1-cni-binary-copy\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.253696 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253516    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5dd3564c-d0e5-444f-9417-4d23ea2951e1-multus-daemon-config\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.253696 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253542    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-var-lib-cni-multus\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.253696 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253568    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-socket-dir\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.253696 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253592    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-etc-selinux\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.253696 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253635    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzcsj\" (UniqueName: \"kubernetes.io/projected/98dd5ea5-e96e-406a-956c-200a1e0314d1-kube-api-access-vzcsj\") pod \"node-ca-d25h7\" (UID: \"98dd5ea5-e96e-406a-956c-200a1e0314d1\") " pod="openshift-image-registry/node-ca-d25h7"
Jul 31 23:46:19.253696 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253689    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-os-release\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.254100 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253724    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-etc-kubernetes\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.254100 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253749    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df98k\" (UniqueName: \"kubernetes.io/projected/5dd3564c-d0e5-444f-9417-4d23ea2951e1-kube-api-access-df98k\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.254100 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253773    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f85fb808-2a50-499e-a3a8-3aa1c33f69a2-tmp-dir\") pod \"node-resolver-twdx5\" (UID: \"f85fb808-2a50-499e-a3a8-3aa1c33f69a2\") " pod="openshift-dns/node-resolver-twdx5"
Jul 31 23:46:19.254100 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253805    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5c4j\" (UniqueName: \"kubernetes.io/projected/f85fb808-2a50-499e-a3a8-3aa1c33f69a2-kube-api-access-c5c4j\") pod \"node-resolver-twdx5\" (UID: \"f85fb808-2a50-499e-a3a8-3aa1c33f69a2\") " pod="openshift-dns/node-resolver-twdx5"
Jul 31 23:46:19.254100 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/98dd5ea5-e96e-406a-956c-200a1e0314d1-serviceca\") pod \"node-ca-d25h7\" (UID: \"98dd5ea5-e96e-406a-956c-200a1e0314d1\") " pod="openshift-image-registry/node-ca-d25h7"
Jul 31 23:46:19.254100 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253876    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-system-cni-dir\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.254100 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253913    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-cnibin\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.254100 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253936    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-hostroot\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.254100 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.253989    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-device-dir\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.254100 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.254050    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-multus-socket-dir-parent\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.254100 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.254090    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-var-lib-cni-bin\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.254575 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.254148    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-run-multus-certs\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.254575 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.254175    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-run-k8s-cni-cncf-io\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.254575 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.254202    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-multus-conf-dir\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.254575 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.254237    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98dd5ea5-e96e-406a-956c-200a1e0314d1-host\") pod \"node-ca-d25h7\" (UID: \"98dd5ea5-e96e-406a-956c-200a1e0314d1\") " pod="openshift-image-registry/node-ca-d25h7"
Jul 31 23:46:19.254575 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.254279    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-multus-cni-dir\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.254575 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.254317    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-run-netns\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.254575 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.254349    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.254575 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.254392    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-registration-dir\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.257843 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.257826    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.257920 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.257904    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 31 23:46:19.258110 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.257826    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 31 23:46:19.259125 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.258258    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 31 23:46:19.259224 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.258319    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zqdpm\""
Jul 31 23:46:19.259285 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.258613    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zqrmw"
Jul 31 23:46:19.259380 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.258707    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 31 23:46:19.261489 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.261350    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 31 23:46:19.261737 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.261719    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-r9mhn\""
Jul 31 23:46:19.261825 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.261767    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:19.261886 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:19.261836    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:19.262181 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.262163    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 31 23:46:19.262575 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.262554    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 31 23:46:19.262684 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.262564    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9v2lp\""
Jul 31 23:46:19.263086 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.263065    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 31 23:46:19.264157 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.264060    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:19.264157 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:19.264122    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:19.289389 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.289356    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2cpgr"
Jul 31 23:46:19.298147 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.297854    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2cpgr"
Jul 31 23:46:19.341179 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.341147    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 31 23:46:19.355175 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355140    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-multus-cni-dir\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.355345 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355188    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bb83a5d0-9c9a-4d84-8562-560971cad828-konnectivity-ca\") pod \"konnectivity-agent-zqrmw\" (UID: \"bb83a5d0-9c9a-4d84-8562-560971cad828\") " pod="kube-system/konnectivity-agent-zqrmw"
Jul 31 23:46:19.355345 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355214    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-log-socket\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.355345 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355238    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c600206-7804-41f1-ae49-de6c40a4d2e0-iptables-alerter-script\") pod \"iptables-alerter-c8qbc\" (UID: \"6c600206-7804-41f1-ae49-de6c40a4d2e0\") " pod="openshift-network-operator/iptables-alerter-c8qbc"
Jul 31 23:46:19.355345 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355261    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-sysctl-d\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.355345 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355283    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-var-lib-kubelet\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.355345 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355305    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-sys-fs\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.355345 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355328    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-slash\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.355719 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355351    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-ovnkube-script-lib\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.355719 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355376    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9da928a7-a1b0-4b64-a8a0-0e007b337734-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.355719 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355404    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret\") pod \"global-pull-secret-syncer-4k4dl\" (UID: \"6c1ae62a-0b92-4205-a4bb-5d266930b743\") " pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:19.355719 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355429    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5dd3564c-d0e5-444f-9417-4d23ea2951e1-cni-binary-copy\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.355719 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355456    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.355719 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355482    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcnt7\" (UniqueName: \"kubernetes.io/projected/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-kube-api-access-bcnt7\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.355719 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76w94\" (UniqueName: \"kubernetes.io/projected/9da928a7-a1b0-4b64-a8a0-0e007b337734-kube-api-access-76w94\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.355719 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355533    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-lib-modules\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.355719 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355562    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-etc-selinux\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.355719 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355591    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzcsj\" (UniqueName: \"kubernetes.io/projected/98dd5ea5-e96e-406a-956c-200a1e0314d1-kube-api-access-vzcsj\") pod \"node-ca-d25h7\" (UID: \"98dd5ea5-e96e-406a-956c-200a1e0314d1\") " pod="openshift-image-registry/node-ca-d25h7"
Jul 31 23:46:19.355719 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355619    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9da928a7-a1b0-4b64-a8a0-0e007b337734-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.355719 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355644    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5knxr\" (UniqueName: \"kubernetes.io/projected/6c600206-7804-41f1-ae49-de6c40a4d2e0-kube-api-access-5knxr\") pod \"iptables-alerter-c8qbc\" (UID: \"6c600206-7804-41f1-ae49-de6c40a4d2e0\") " pod="openshift-network-operator/iptables-alerter-c8qbc"
Jul 31 23:46:19.355719 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355670    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c1ae62a-0b92-4205-a4bb-5d266930b743-kubelet-config\") pod \"global-pull-secret-syncer-4k4dl\" (UID: \"6c1ae62a-0b92-4205-a4bb-5d266930b743\") " pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:19.355719 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f85fb808-2a50-499e-a3a8-3aa1c33f69a2-tmp-dir\") pod \"node-resolver-twdx5\" (UID: \"f85fb808-2a50-499e-a3a8-3aa1c33f69a2\") " pod="openshift-dns/node-resolver-twdx5"
Jul 31 23:46:19.355719 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355724    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5c4j\" (UniqueName: \"kubernetes.io/projected/f85fb808-2a50-499e-a3a8-3aa1c33f69a2-kube-api-access-c5c4j\") pod \"node-resolver-twdx5\" (UID: \"f85fb808-2a50-499e-a3a8-3aa1c33f69a2\") " pod="openshift-dns/node-resolver-twdx5"
Jul 31 23:46:19.356457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355751    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/98dd5ea5-e96e-406a-956c-200a1e0314d1-serviceca\") pod \"node-ca-d25h7\" (UID: \"98dd5ea5-e96e-406a-956c-200a1e0314d1\") " pod="openshift-image-registry/node-ca-d25h7"
Jul 31 23:46:19.356457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355779    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bb83a5d0-9c9a-4d84-8562-560971cad828-agent-certs\") pod \"konnectivity-agent-zqrmw\" (UID: \"bb83a5d0-9c9a-4d84-8562-560971cad828\") " pod="kube-system/konnectivity-agent-zqrmw"
Jul 31 23:46:19.356457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355806    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-kubelet\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.356457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355842    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-systemd-units\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.356457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355869    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9da928a7-a1b0-4b64-a8a0-0e007b337734-cnibin\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.356457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355883    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-var-lib-kubelet\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.356457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355899    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-etc-selinux\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.356457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355903    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-device-dir\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.356457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355955    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9da928a7-a1b0-4b64-a8a0-0e007b337734-os-release\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.356457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.355959    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-multus-cni-dir\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.356457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.356102    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-sys-fs\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.356995 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.356586    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f85fb808-2a50-499e-a3a8-3aa1c33f69a2-tmp-dir\") pod \"node-resolver-twdx5\" (UID: \"f85fb808-2a50-499e-a3a8-3aa1c33f69a2\") " pod="openshift-dns/node-resolver-twdx5"
Jul 31 23:46:19.356995 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.356655    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-sysconfig\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.356995 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.356685    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/98dd5ea5-e96e-406a-956c-200a1e0314d1-serviceca\") pod \"node-ca-d25h7\" (UID: \"98dd5ea5-e96e-406a-956c-200a1e0314d1\") " pod="openshift-image-registry/node-ca-d25h7"
Jul 31 23:46:19.356995 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.356720    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-kubernetes\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.356995 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.356714    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-device-dir\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.356995 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.356770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-sysctl-conf\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.356995 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.356773    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5dd3564c-d0e5-444f-9417-4d23ea2951e1-cni-binary-copy\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.356995 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.356891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-multus-socket-dir-parent\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.356995 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.356923    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-run-multus-certs\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.356995 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.356950    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9da928a7-a1b0-4b64-a8a0-0e007b337734-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357004    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c1ae62a-0b92-4205-a4bb-5d266930b743-dbus\") pod \"global-pull-secret-syncer-4k4dl\" (UID: \"6c1ae62a-0b92-4205-a4bb-5d266930b743\") " pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357024    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-run-multus-certs\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-run-k8s-cni-cncf-io\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357073    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-multus-conf-dir\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357100    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98dd5ea5-e96e-406a-956c-200a1e0314d1-host\") pod \"node-ca-d25h7\" (UID: \"98dd5ea5-e96e-406a-956c-200a1e0314d1\") " pod="openshift-image-registry/node-ca-d25h7"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357129    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs\") pod \"network-metrics-daemon-tn7kj\" (UID: \"75c8e3ed-6587-4ef9-b3bb-562bf74fec5b\") " pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357138    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-multus-conf-dir\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357138    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-multus-socket-dir-parent\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357154    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-ovnkube-config\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357075    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-run-k8s-cni-cncf-io\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357184    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9da928a7-a1b0-4b64-a8a0-0e007b337734-cni-binary-copy\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357188    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98dd5ea5-e96e-406a-956c-200a1e0314d1-host\") pod \"node-ca-d25h7\" (UID: \"98dd5ea5-e96e-406a-956c-200a1e0314d1\") " pod="openshift-image-registry/node-ca-d25h7"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357212    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8685c5a-294f-4e78-9504-085286bd468a-tmp\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357239    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-run-netns\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357267    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357294    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-registration-dir\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.357523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357320    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-run-ovn\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357335    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-run-netns\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357347    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-run-ovn-kubernetes\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357415    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357470    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-registration-dir\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357481    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-ovn-node-metrics-cert\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357520    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzfcq\" (UniqueName: \"kubernetes.io/projected/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-kube-api-access-bzfcq\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357549    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f85fb808-2a50-499e-a3a8-3aa1c33f69a2-hosts-file\") pod \"node-resolver-twdx5\" (UID: \"f85fb808-2a50-499e-a3a8-3aa1c33f69a2\") " pod="openshift-dns/node-resolver-twdx5"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357579    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-run-systemd\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357660    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f85fb808-2a50-499e-a3a8-3aa1c33f69a2-hosts-file\") pod \"node-resolver-twdx5\" (UID: \"f85fb808-2a50-499e-a3a8-3aa1c33f69a2\") " pod="openshift-dns/node-resolver-twdx5"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357784    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-modprobe-d\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357822    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-run\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357853    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-var-lib-kubelet\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357886    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5dd3564c-d0e5-444f-9417-4d23ea2951e1-multus-daemon-config\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.357918    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-run-openvswitch\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358034    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c600206-7804-41f1-ae49-de6c40a4d2e0-host-slash\") pod \"iptables-alerter-c8qbc\" (UID: \"6c600206-7804-41f1-ae49-de6c40a4d2e0\") " pod="openshift-network-operator/iptables-alerter-c8qbc"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-host\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.358333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358095    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-var-lib-cni-multus\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358124    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-socket-dir\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358149    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-var-lib-openvswitch\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358157    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-var-lib-cni-multus\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358174    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-node-log\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358231    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8685c5a-294f-4e78-9504-085286bd468a-etc-tuned\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358259    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-os-release\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358283    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-socket-dir\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358286    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-etc-kubernetes\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358314    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df98k\" (UniqueName: \"kubernetes.io/projected/5dd3564c-d0e5-444f-9417-4d23ea2951e1-kube-api-access-df98k\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358344    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-run-netns\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358364    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-os-release\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358378    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87ptm\" (UniqueName: \"kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm\") pod \"network-check-target-frb4r\" (UID: \"1cc3f661-c981-4802-964e-632fab8107c6\") " pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358424    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5dd3564c-d0e5-444f-9417-4d23ea2951e1-multus-daemon-config\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358425    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-system-cni-dir\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358466    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-cnibin\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358480    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-system-cni-dir\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-hostroot\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.359105 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358519    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-hostroot\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.359851 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358521    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtksv\" (UniqueName: \"kubernetes.io/projected/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-kube-api-access-xtksv\") pod \"network-metrics-daemon-tn7kj\" (UID: \"75c8e3ed-6587-4ef9-b3bb-562bf74fec5b\") " pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:19.359851 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358548    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-etc-kubernetes\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.359851 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358549    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-cni-bin\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.359851 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358589    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-cnibin\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.359851 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358590    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-env-overrides\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.359851 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358618    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-var-lib-cni-bin\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.359851 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358644    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9da928a7-a1b0-4b64-a8a0-0e007b337734-system-cni-dir\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.359851 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358671    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-sys\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.359851 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358685    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5dd3564c-d0e5-444f-9417-4d23ea2951e1-host-var-lib-cni-bin\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.359851 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-etc-openvswitch\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.359851 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358728    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-cni-netd\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.359851 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358765    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-systemd\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.359851 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.358799    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdqdd\" (UniqueName: \"kubernetes.io/projected/d8685c5a-294f-4e78-9504-085286bd468a-kube-api-access-kdqdd\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.362921 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.362891    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 31 23:46:19.366848 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.366819    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5c4j\" (UniqueName: \"kubernetes.io/projected/f85fb808-2a50-499e-a3a8-3aa1c33f69a2-kube-api-access-c5c4j\") pod \"node-resolver-twdx5\" (UID: \"f85fb808-2a50-499e-a3a8-3aa1c33f69a2\") " pod="openshift-dns/node-resolver-twdx5"
Jul 31 23:46:19.367460 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.367439    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzcsj\" (UniqueName: \"kubernetes.io/projected/98dd5ea5-e96e-406a-956c-200a1e0314d1-kube-api-access-vzcsj\") pod \"node-ca-d25h7\" (UID: \"98dd5ea5-e96e-406a-956c-200a1e0314d1\") " pod="openshift-image-registry/node-ca-d25h7"
Jul 31 23:46:19.369131 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.369103    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzfcq\" (UniqueName: \"kubernetes.io/projected/6ba5f015-bfbc-45c6-9e7f-255c700ecdb6-kube-api-access-bzfcq\") pod \"aws-ebs-csi-driver-node-p44lf\" (UID: \"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.369960 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.369934    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df98k\" (UniqueName: \"kubernetes.io/projected/5dd3564c-d0e5-444f-9417-4d23ea2951e1-kube-api-access-df98k\") pod \"multus-hftd5\" (UID: \"5dd3564c-d0e5-444f-9417-4d23ea2951e1\") " pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.388516 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.388442    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-141.ec2.internal" event={"ID":"109099d262acad3a59990c8222269413","Type":"ContainerStarted","Data":"2b809fb7d8e3da0b22e467d7897361d49c4ad2eec5af70efbb8cff486d665806"}
Jul 31 23:46:19.390075 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.390007    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal" event={"ID":"8a14086636be88f16b8a69fdf15ede03","Type":"ContainerStarted","Data":"3a09ef6fa054323cc92335f7cd49ea0ee62c66a53001b02709674b5f73fb441f"}
Jul 31 23:46:19.459559 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459519    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-etc-openvswitch\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.459769 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459578    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-cni-netd\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.459769 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459607    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-systemd\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.459769 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459636    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdqdd\" (UniqueName: \"kubernetes.io/projected/d8685c5a-294f-4e78-9504-085286bd468a-kube-api-access-kdqdd\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.459769 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459667    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bb83a5d0-9c9a-4d84-8562-560971cad828-konnectivity-ca\") pod \"konnectivity-agent-zqrmw\" (UID: \"bb83a5d0-9c9a-4d84-8562-560971cad828\") " pod="kube-system/konnectivity-agent-zqrmw"
Jul 31 23:46:19.459769 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459689    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-systemd\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.459769 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459695    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-cni-netd\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.459769 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459691    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-log-socket\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.459769 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459634    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-etc-openvswitch\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.459769 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459723    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-log-socket\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.459769 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c600206-7804-41f1-ae49-de6c40a4d2e0-iptables-alerter-script\") pod \"iptables-alerter-c8qbc\" (UID: \"6c600206-7804-41f1-ae49-de6c40a4d2e0\") " pod="openshift-network-operator/iptables-alerter-c8qbc"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459797    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-sysctl-d\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459829    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-slash\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-ovnkube-script-lib\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459881    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9da928a7-a1b0-4b64-a8a0-0e007b337734-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459913    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret\") pod \"global-pull-secret-syncer-4k4dl\" (UID: \"6c1ae62a-0b92-4205-a4bb-5d266930b743\") " pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.459965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcnt7\" (UniqueName: \"kubernetes.io/projected/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-kube-api-access-bcnt7\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460007    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76w94\" (UniqueName: \"kubernetes.io/projected/9da928a7-a1b0-4b64-a8a0-0e007b337734-kube-api-access-76w94\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460031    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-lib-modules\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460060    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9da928a7-a1b0-4b64-a8a0-0e007b337734-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5knxr\" (UniqueName: \"kubernetes.io/projected/6c600206-7804-41f1-ae49-de6c40a4d2e0-kube-api-access-5knxr\") pod \"iptables-alerter-c8qbc\" (UID: \"6c600206-7804-41f1-ae49-de6c40a4d2e0\") " pod="openshift-network-operator/iptables-alerter-c8qbc"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460113    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c1ae62a-0b92-4205-a4bb-5d266930b743-kubelet-config\") pod \"global-pull-secret-syncer-4k4dl\" (UID: \"6c1ae62a-0b92-4205-a4bb-5d266930b743\") " pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460142    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bb83a5d0-9c9a-4d84-8562-560971cad828-agent-certs\") pod \"konnectivity-agent-zqrmw\" (UID: \"bb83a5d0-9c9a-4d84-8562-560971cad828\") " pod="kube-system/konnectivity-agent-zqrmw"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460167    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-kubelet\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460195    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-systemd-units\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460221    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9da928a7-a1b0-4b64-a8a0-0e007b337734-cnibin\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.460319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9da928a7-a1b0-4b64-a8a0-0e007b337734-os-release\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460275    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-sysconfig\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460301    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-kubernetes\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460325    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-sysctl-conf\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460354    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9da928a7-a1b0-4b64-a8a0-0e007b337734-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460378    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c1ae62a-0b92-4205-a4bb-5d266930b743-dbus\") pod \"global-pull-secret-syncer-4k4dl\" (UID: \"6c1ae62a-0b92-4205-a4bb-5d266930b743\") " pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460408    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c1ae62a-0b92-4205-a4bb-5d266930b743-kubelet-config\") pod \"global-pull-secret-syncer-4k4dl\" (UID: \"6c1ae62a-0b92-4205-a4bb-5d266930b743\") " pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460410    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs\") pod \"network-metrics-daemon-tn7kj\" (UID: \"75c8e3ed-6587-4ef9-b3bb-562bf74fec5b\") " pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-ovnkube-config\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460489    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9da928a7-a1b0-4b64-a8a0-0e007b337734-cni-binary-copy\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:19.460505    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460515    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8685c5a-294f-4e78-9504-085286bd468a-tmp\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460547    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-run-ovn\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460566    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c600206-7804-41f1-ae49-de6c40a4d2e0-iptables-alerter-script\") pod \"iptables-alerter-c8qbc\" (UID: \"6c600206-7804-41f1-ae49-de6c40a4d2e0\") " pod="openshift-network-operator/iptables-alerter-c8qbc"
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460582    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-ovnkube-script-lib\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460573    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-run-ovn-kubernetes\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:19.460621    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs podName:75c8e3ed-6587-4ef9-b3bb-562bf74fec5b nodeName:}" failed. No retries permitted until 2026-07-31 23:46:19.960580689 +0000 UTC m=+3.140037414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs") pod "network-metrics-daemon-tn7kj" (UID: "75c8e3ed-6587-4ef9-b3bb-562bf74fec5b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:19.461228 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460651    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-slash\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460717    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-sysconfig\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460803    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-kubernetes\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460950    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-sysctl-conf\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461118    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9da928a7-a1b0-4b64-a8a0-0e007b337734-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461126    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c1ae62a-0b92-4205-a4bb-5d266930b743-dbus\") pod \"global-pull-secret-syncer-4k4dl\" (UID: \"6c1ae62a-0b92-4205-a4bb-5d266930b743\") " pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.460355    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bb83a5d0-9c9a-4d84-8562-560971cad828-konnectivity-ca\") pod \"konnectivity-agent-zqrmw\" (UID: \"bb83a5d0-9c9a-4d84-8562-560971cad828\") " pod="kube-system/konnectivity-agent-zqrmw"
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461186    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-systemd-units\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461196    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9da928a7-a1b0-4b64-a8a0-0e007b337734-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461233    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-kubelet\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:19.461263    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461275    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461292    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-ovnkube-config\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461345    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9da928a7-a1b0-4b64-a8a0-0e007b337734-os-release\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461471    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-lib-modules\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461479    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-run-ovn\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461584    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-run-ovn-kubernetes\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.462227 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:19.461658    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret podName:6c1ae62a-0b92-4205-a4bb-5d266930b743 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:19.961638261 +0000 UTC m=+3.141094992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret") pod "global-pull-secret-syncer-4k4dl" (UID: "6c1ae62a-0b92-4205-a4bb-5d266930b743") : object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461686    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-ovn-node-metrics-cert\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461718    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-run-systemd\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461744    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-modprobe-d\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461771    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-run\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461777    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9da928a7-a1b0-4b64-a8a0-0e007b337734-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461796    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-var-lib-kubelet\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461824    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-run-openvswitch\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461854    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c600206-7804-41f1-ae49-de6c40a4d2e0-host-slash\") pod \"iptables-alerter-c8qbc\" (UID: \"6c600206-7804-41f1-ae49-de6c40a4d2e0\") " pod="openshift-network-operator/iptables-alerter-c8qbc"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461878    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-host\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461905    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-var-lib-openvswitch\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461916    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-sysctl-d\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461200    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9da928a7-a1b0-4b64-a8a0-0e007b337734-cnibin\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461960    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-node-log\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461927    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-node-log\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.461977    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-var-lib-kubelet\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.462001    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-host\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.462008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8685c5a-294f-4e78-9504-085286bd468a-etc-tuned\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.472664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.462013    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c600206-7804-41f1-ae49-de6c40a4d2e0-host-slash\") pod \"iptables-alerter-c8qbc\" (UID: \"6c600206-7804-41f1-ae49-de6c40a4d2e0\") " pod="openshift-network-operator/iptables-alerter-c8qbc"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.462045    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-run-netns\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.462064    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-var-lib-openvswitch\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.462074    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87ptm\" (UniqueName: \"kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm\") pod \"network-check-target-frb4r\" (UID: \"1cc3f661-c981-4802-964e-632fab8107c6\") " pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.462080    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-run-openvswitch\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.462109    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtksv\" (UniqueName: \"kubernetes.io/projected/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-kube-api-access-xtksv\") pod \"network-metrics-daemon-tn7kj\" (UID: \"75c8e3ed-6587-4ef9-b3bb-562bf74fec5b\") " pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.462124    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-run-netns\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.462135    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-cni-bin\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.462160    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-env-overrides\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.462185    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9da928a7-a1b0-4b64-a8a0-0e007b337734-system-cni-dir\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.462208    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-sys\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.462287    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-sys\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.463405    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-etc-modprobe-d\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.463641    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9da928a7-a1b0-4b64-a8a0-0e007b337734-cni-binary-copy\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.463668    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-host-cni-bin\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.464085    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-env-overrides\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.464126    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bb83a5d0-9c9a-4d84-8562-560971cad828-agent-certs\") pod \"konnectivity-agent-zqrmw\" (UID: \"bb83a5d0-9c9a-4d84-8562-560971cad828\") " pod="kube-system/konnectivity-agent-zqrmw"
Jul 31 23:46:19.473457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.464154    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9da928a7-a1b0-4b64-a8a0-0e007b337734-system-cni-dir\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.474300 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.464222    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8685c5a-294f-4e78-9504-085286bd468a-run\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.474300 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.464251    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-run-systemd\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.474300 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:19.469007    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 23:46:19.474300 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:19.469055    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 23:46:19.474300 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:19.469071    2581 projected.go:196] Error preparing data for projected volume kube-api-access-87ptm for pod openshift-network-diagnostics/network-check-target-frb4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:19.474300 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:19.469152    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm podName:1cc3f661-c981-4802-964e-632fab8107c6 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:19.969132041 +0000 UTC m=+3.148588771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-87ptm" (UniqueName: "kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm") pod "network-check-target-frb4r" (UID: "1cc3f661-c981-4802-964e-632fab8107c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:19.474300 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.470430    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdqdd\" (UniqueName: \"kubernetes.io/projected/d8685c5a-294f-4e78-9504-085286bd468a-kube-api-access-kdqdd\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.474300 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.470436    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5knxr\" (UniqueName: \"kubernetes.io/projected/6c600206-7804-41f1-ae49-de6c40a4d2e0-kube-api-access-5knxr\") pod \"iptables-alerter-c8qbc\" (UID: \"6c600206-7804-41f1-ae49-de6c40a4d2e0\") " pod="openshift-network-operator/iptables-alerter-c8qbc"
Jul 31 23:46:19.474300 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.471839    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8685c5a-294f-4e78-9504-085286bd468a-tmp\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.474300 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.472028    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcnt7\" (UniqueName: \"kubernetes.io/projected/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-kube-api-access-bcnt7\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.474300 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.472537    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76w94\" (UniqueName: \"kubernetes.io/projected/9da928a7-a1b0-4b64-a8a0-0e007b337734-kube-api-access-76w94\") pod \"multus-additional-cni-plugins-q6bv6\" (UID: \"9da928a7-a1b0-4b64-a8a0-0e007b337734\") " pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.474300 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.474297    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8685c5a-294f-4e78-9504-085286bd468a-etc-tuned\") pod \"tuned-dpmjk\" (UID: \"d8685c5a-294f-4e78-9504-085286bd468a\") " pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.474734 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.474454    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4869dda6-8fe5-4c60-abc7-f16ce429b2d4-ovn-node-metrics-cert\") pod \"ovnkube-node-5sqvw\" (UID: \"4869dda6-8fe5-4c60-abc7-f16ce429b2d4\") " pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.474734 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.474707    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtksv\" (UniqueName: \"kubernetes.io/projected/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-kube-api-access-xtksv\") pod \"network-metrics-daemon-tn7kj\" (UID: \"75c8e3ed-6587-4ef9-b3bb-562bf74fec5b\") " pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:19.535409 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.535375    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 23:46:19.542786 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.542748    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hftd5"
Jul 31 23:46:19.550037 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.550017    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf"
Jul 31 23:46:19.561926 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.561893    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-twdx5"
Jul 31 23:46:19.574075 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.574046    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d25h7"
Jul 31 23:46:19.581218 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.581193    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:19.591213 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.591189    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q6bv6"
Jul 31 23:46:19.599262 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.599218    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c8qbc"
Jul 31 23:46:19.608120 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.608091    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dpmjk"
Jul 31 23:46:19.615220 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.615197    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zqrmw"
Jul 31 23:46:19.967356 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.967272    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs\") pod \"network-metrics-daemon-tn7kj\" (UID: \"75c8e3ed-6587-4ef9-b3bb-562bf74fec5b\") " pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:19.967522 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:19.967355    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret\") pod \"global-pull-secret-syncer-4k4dl\" (UID: \"6c1ae62a-0b92-4205-a4bb-5d266930b743\") " pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:19.967522 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:19.967470    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:19.967522 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:19.967475    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:19.967687 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:19.967547    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret podName:6c1ae62a-0b92-4205-a4bb-5d266930b743 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:20.967525926 +0000 UTC m=+4.146982667 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret") pod "global-pull-secret-syncer-4k4dl" (UID: "6c1ae62a-0b92-4205-a4bb-5d266930b743") : object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:19.967687 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:19.967568    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs podName:75c8e3ed-6587-4ef9-b3bb-562bf74fec5b nodeName:}" failed. No retries permitted until 2026-07-31 23:46:20.967556981 +0000 UTC m=+4.147013711 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs") pod "network-metrics-daemon-tn7kj" (UID: "75c8e3ed-6587-4ef9-b3bb-562bf74fec5b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:20.068321 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:20.068279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87ptm\" (UniqueName: \"kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm\") pod \"network-check-target-frb4r\" (UID: \"1cc3f661-c981-4802-964e-632fab8107c6\") " pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:20.068517 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:20.068468    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 23:46:20.068517 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:20.068491    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 23:46:20.068517 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:20.068501    2581 projected.go:196] Error preparing data for projected volume kube-api-access-87ptm for pod openshift-network-diagnostics/network-check-target-frb4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:20.068649 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:20.068566    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm podName:1cc3f661-c981-4802-964e-632fab8107c6 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:21.068549944 +0000 UTC m=+4.248006678 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-87ptm" (UniqueName: "kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm") pod "network-check-target-frb4r" (UID: "1cc3f661-c981-4802-964e-632fab8107c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:20.302487 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:20.302387    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 23:41:19 +0000 UTC" deadline="2028-05-09 14:27:27.495855326 +0000 UTC"
Jul 31 23:46:20.302487 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:20.302442    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15542h41m7.193417382s"
Jul 31 23:46:20.361157 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:20.361123    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9da928a7_a1b0_4b64_a8a0_0e007b337734.slice/crio-2d7f66456ced47451827926a4caf4176555a31523f2df40defdf3dfcbcd28239 WatchSource:0}: Error finding container 2d7f66456ced47451827926a4caf4176555a31523f2df40defdf3dfcbcd28239: Status 404 returned error can't find the container with id 2d7f66456ced47451827926a4caf4176555a31523f2df40defdf3dfcbcd28239
Jul 31 23:46:20.362847 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:20.362822    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c600206_7804_41f1_ae49_de6c40a4d2e0.slice/crio-0b78a6f1d22cb3361a99bbddc86ea601a801e91c1bd795107faa797b27c45767 WatchSource:0}: Error finding container 0b78a6f1d22cb3361a99bbddc86ea601a801e91c1bd795107faa797b27c45767: Status 404 returned error can't find the container with id 0b78a6f1d22cb3361a99bbddc86ea601a801e91c1bd795107faa797b27c45767
Jul 31 23:46:20.363876 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:20.363853    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dd3564c_d0e5_444f_9417_4d23ea2951e1.slice/crio-98250b3779efe09061d40d6e6b1c6c7f0f831f242c642a3e2c68289483e9dfe5 WatchSource:0}: Error finding container 98250b3779efe09061d40d6e6b1c6c7f0f831f242c642a3e2c68289483e9dfe5: Status 404 returned error can't find the container with id 98250b3779efe09061d40d6e6b1c6c7f0f831f242c642a3e2c68289483e9dfe5
Jul 31 23:46:20.367308 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:20.367280    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4869dda6_8fe5_4c60_abc7_f16ce429b2d4.slice/crio-52c85441e265ed4962c534acc36d749a541a6cd40cb1c5dee84de769b1aa9847 WatchSource:0}: Error finding container 52c85441e265ed4962c534acc36d749a541a6cd40cb1c5dee84de769b1aa9847: Status 404 returned error can't find the container with id 52c85441e265ed4962c534acc36d749a541a6cd40cb1c5dee84de769b1aa9847
Jul 31 23:46:20.368791 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:20.368731    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98dd5ea5_e96e_406a_956c_200a1e0314d1.slice/crio-82e67d15ef59a16fa90801f4f1f412684db02d34929609304807c92a4254ebd9 WatchSource:0}: Error finding container 82e67d15ef59a16fa90801f4f1f412684db02d34929609304807c92a4254ebd9: Status 404 returned error can't find the container with id 82e67d15ef59a16fa90801f4f1f412684db02d34929609304807c92a4254ebd9
Jul 31 23:46:20.369761 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:20.369524    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf85fb808_2a50_499e_a3a8_3aa1c33f69a2.slice/crio-a38ac23e8998973994d202f9cd18a7f0d49550119c8fc80b8eae267b4f62339c WatchSource:0}: Error finding container a38ac23e8998973994d202f9cd18a7f0d49550119c8fc80b8eae267b4f62339c: Status 404 returned error can't find the container with id a38ac23e8998973994d202f9cd18a7f0d49550119c8fc80b8eae267b4f62339c
Jul 31 23:46:20.370611 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:20.370572    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8685c5a_294f_4e78_9504_085286bd468a.slice/crio-a7acd98c66a171c549207ab88e4d2f9aa3cc549fad8b2e98493fcdf0849249ff WatchSource:0}: Error finding container a7acd98c66a171c549207ab88e4d2f9aa3cc549fad8b2e98493fcdf0849249ff: Status 404 returned error can't find the container with id a7acd98c66a171c549207ab88e4d2f9aa3cc549fad8b2e98493fcdf0849249ff
Jul 31 23:46:20.372854 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:20.372830    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb83a5d0_9c9a_4d84_8562_560971cad828.slice/crio-ecdc66e2125f1752ab2dbf8e16f53c3a7eac674d6ee75cd19a4391c2ddef0dfd WatchSource:0}: Error finding container ecdc66e2125f1752ab2dbf8e16f53c3a7eac674d6ee75cd19a4391c2ddef0dfd: Status 404 returned error can't find the container with id ecdc66e2125f1752ab2dbf8e16f53c3a7eac674d6ee75cd19a4391c2ddef0dfd
Jul 31 23:46:20.373181 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:20.373155    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ba5f015_bfbc_45c6_9e7f_255c700ecdb6.slice/crio-d680360703aaf4ee16d86e8286faf11ccf882dd1c055539e6b62d09bfe1282a5 WatchSource:0}: Error finding container d680360703aaf4ee16d86e8286faf11ccf882dd1c055539e6b62d09bfe1282a5: Status 404 returned error can't find the container with id d680360703aaf4ee16d86e8286faf11ccf882dd1c055539e6b62d09bfe1282a5
Jul 31 23:46:20.382793 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:20.382765    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:20.382990 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:20.382963    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:20.393333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:20.393305    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c8qbc" event={"ID":"6c600206-7804-41f1-ae49-de6c40a4d2e0","Type":"ContainerStarted","Data":"0b78a6f1d22cb3361a99bbddc86ea601a801e91c1bd795107faa797b27c45767"}
Jul 31 23:46:20.394448 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:20.394416    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf" event={"ID":"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6","Type":"ContainerStarted","Data":"d680360703aaf4ee16d86e8286faf11ccf882dd1c055539e6b62d09bfe1282a5"}
Jul 31 23:46:20.395825 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:20.395788    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dpmjk" event={"ID":"d8685c5a-294f-4e78-9504-085286bd468a","Type":"ContainerStarted","Data":"a7acd98c66a171c549207ab88e4d2f9aa3cc549fad8b2e98493fcdf0849249ff"}
Jul 31 23:46:20.399350 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:20.399322    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-twdx5" event={"ID":"f85fb808-2a50-499e-a3a8-3aa1c33f69a2","Type":"ContainerStarted","Data":"a38ac23e8998973994d202f9cd18a7f0d49550119c8fc80b8eae267b4f62339c"}
Jul 31 23:46:20.400854 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:20.400830    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw" event={"ID":"4869dda6-8fe5-4c60-abc7-f16ce429b2d4","Type":"ContainerStarted","Data":"52c85441e265ed4962c534acc36d749a541a6cd40cb1c5dee84de769b1aa9847"}
Jul 31 23:46:20.401916 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:20.401891    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6bv6" event={"ID":"9da928a7-a1b0-4b64-a8a0-0e007b337734","Type":"ContainerStarted","Data":"2d7f66456ced47451827926a4caf4176555a31523f2df40defdf3dfcbcd28239"}
Jul 31 23:46:20.405308 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:20.405280    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zqrmw" event={"ID":"bb83a5d0-9c9a-4d84-8562-560971cad828","Type":"ContainerStarted","Data":"ecdc66e2125f1752ab2dbf8e16f53c3a7eac674d6ee75cd19a4391c2ddef0dfd"}
Jul 31 23:46:20.409009 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:20.408985    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d25h7" event={"ID":"98dd5ea5-e96e-406a-956c-200a1e0314d1","Type":"ContainerStarted","Data":"82e67d15ef59a16fa90801f4f1f412684db02d34929609304807c92a4254ebd9"}
Jul 31 23:46:20.411577 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:20.411555    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hftd5" event={"ID":"5dd3564c-d0e5-444f-9417-4d23ea2951e1","Type":"ContainerStarted","Data":"98250b3779efe09061d40d6e6b1c6c7f0f831f242c642a3e2c68289483e9dfe5"}
Jul 31 23:46:20.976715 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:20.976347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret\") pod \"global-pull-secret-syncer-4k4dl\" (UID: \"6c1ae62a-0b92-4205-a4bb-5d266930b743\") " pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:20.976715 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:20.976697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs\") pod \"network-metrics-daemon-tn7kj\" (UID: \"75c8e3ed-6587-4ef9-b3bb-562bf74fec5b\") " pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:20.976970 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:20.976926    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:20.977056 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:20.976995    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs podName:75c8e3ed-6587-4ef9-b3bb-562bf74fec5b nodeName:}" failed. No retries permitted until 2026-07-31 23:46:22.976975263 +0000 UTC m=+6.156431996 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs") pod "network-metrics-daemon-tn7kj" (UID: "75c8e3ed-6587-4ef9-b3bb-562bf74fec5b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:20.977574 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:20.977479    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:20.977574 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:20.977547    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret podName:6c1ae62a-0b92-4205-a4bb-5d266930b743 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:22.977529401 +0000 UTC m=+6.156986128 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret") pod "global-pull-secret-syncer-4k4dl" (UID: "6c1ae62a-0b92-4205-a4bb-5d266930b743") : object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:21.077698 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:21.077656    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87ptm\" (UniqueName: \"kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm\") pod \"network-check-target-frb4r\" (UID: \"1cc3f661-c981-4802-964e-632fab8107c6\") " pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:21.077902 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:21.077887    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 23:46:21.077966 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:21.077910    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 23:46:21.077966 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:21.077924    2581 projected.go:196] Error preparing data for projected volume kube-api-access-87ptm for pod openshift-network-diagnostics/network-check-target-frb4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:21.078126 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:21.077986    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm podName:1cc3f661-c981-4802-964e-632fab8107c6 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:23.077964819 +0000 UTC m=+6.257421568 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-87ptm" (UniqueName: "kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm") pod "network-check-target-frb4r" (UID: "1cc3f661-c981-4802-964e-632fab8107c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:21.303266 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:21.303171    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 23:41:19 +0000 UTC" deadline="2028-04-17 04:50:53.578290156 +0000 UTC"
Jul 31 23:46:21.303266 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:21.303216    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15005h4m32.275079064s"
Jul 31 23:46:21.385081 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:21.385028    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:21.385298 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:21.385100    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:21.385298 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:21.385203    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:21.385430 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:21.385327    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:21.440201 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:21.440162    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-141.ec2.internal" event={"ID":"109099d262acad3a59990c8222269413","Type":"ContainerStarted","Data":"47b2ce8e1660948ad9320d7e80c0f966d559fa307973f220956e9cce16af97f3"}
Jul 31 23:46:21.452086 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:21.451701    2581 generic.go:356] "Generic (PLEG): container finished" podID="8a14086636be88f16b8a69fdf15ede03" containerID="701da23bde63b54c725ab29a8a54f1c9e23dfcbfbf177cfaad3de24beb96cfd5" exitCode=0
Jul 31 23:46:21.452086 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:21.451757    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal" event={"ID":"8a14086636be88f16b8a69fdf15ede03","Type":"ContainerDied","Data":"701da23bde63b54c725ab29a8a54f1c9e23dfcbfbf177cfaad3de24beb96cfd5"}
Jul 31 23:46:21.456135 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:21.456027    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-141.ec2.internal" podStartSLOduration=2.456006521 podStartE2EDuration="2.456006521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:21.45572361 +0000 UTC m=+4.635180359" watchObservedRunningTime="2026-07-31 23:46:21.456006521 +0000 UTC m=+4.635463260"
Jul 31 23:46:22.382424 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:22.382321    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:22.383048 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:22.382495    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:22.496234 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:22.495778    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal" event={"ID":"8a14086636be88f16b8a69fdf15ede03","Type":"ContainerStarted","Data":"06d4f05439840d6ec9490025ac90abdf39aba7c64e27401e35b94377fa341614"}
Jul 31 23:46:22.523339 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:22.522508    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-141.ec2.internal" podStartSLOduration=3.522483737 podStartE2EDuration="3.522483737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:22.521754186 +0000 UTC m=+5.701210936" watchObservedRunningTime="2026-07-31 23:46:22.522483737 +0000 UTC m=+5.701940487"
Jul 31 23:46:22.993508 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:22.993469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs\") pod \"network-metrics-daemon-tn7kj\" (UID: \"75c8e3ed-6587-4ef9-b3bb-562bf74fec5b\") " pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:22.993720 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:22.993536    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret\") pod \"global-pull-secret-syncer-4k4dl\" (UID: \"6c1ae62a-0b92-4205-a4bb-5d266930b743\") " pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:22.993720 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:22.993646    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:22.993720 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:22.993703    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret podName:6c1ae62a-0b92-4205-a4bb-5d266930b743 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:26.993684726 +0000 UTC m=+10.173141451 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret") pod "global-pull-secret-syncer-4k4dl" (UID: "6c1ae62a-0b92-4205-a4bb-5d266930b743") : object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:22.994171 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:22.994147    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:22.994259 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:22.994219    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs podName:75c8e3ed-6587-4ef9-b3bb-562bf74fec5b nodeName:}" failed. No retries permitted until 2026-07-31 23:46:26.994200746 +0000 UTC m=+10.173657470 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs") pod "network-metrics-daemon-tn7kj" (UID: "75c8e3ed-6587-4ef9-b3bb-562bf74fec5b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:23.095242 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:23.094583    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87ptm\" (UniqueName: \"kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm\") pod \"network-check-target-frb4r\" (UID: \"1cc3f661-c981-4802-964e-632fab8107c6\") " pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:23.095242 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:23.094761    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 23:46:23.095242 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:23.094783    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 23:46:23.095242 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:23.094797    2581 projected.go:196] Error preparing data for projected volume kube-api-access-87ptm for pod openshift-network-diagnostics/network-check-target-frb4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:23.095242 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:23.094864    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm podName:1cc3f661-c981-4802-964e-632fab8107c6 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:27.094843246 +0000 UTC m=+10.274299994 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-87ptm" (UniqueName: "kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm") pod "network-check-target-frb4r" (UID: "1cc3f661-c981-4802-964e-632fab8107c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:23.383820 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:23.383155    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:23.383820 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:23.383299    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:23.384755 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:23.384529    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:23.384755 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:23.384655    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:24.382467 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:24.382398    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:24.382683 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:24.382626    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:25.385774 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:25.385312    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:25.385774 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:25.385359    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:25.385774 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:25.385443    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:25.385774 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:25.385583    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:26.382925 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:26.382749    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:26.383129 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:26.382914    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:27.032613 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:27.032565    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret\") pod \"global-pull-secret-syncer-4k4dl\" (UID: \"6c1ae62a-0b92-4205-a4bb-5d266930b743\") " pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:27.033222 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:27.032645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs\") pod \"network-metrics-daemon-tn7kj\" (UID: \"75c8e3ed-6587-4ef9-b3bb-562bf74fec5b\") " pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:27.033222 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:27.032740    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:27.033222 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:27.032762    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:27.033222 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:27.032820    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs podName:75c8e3ed-6587-4ef9-b3bb-562bf74fec5b nodeName:}" failed. No retries permitted until 2026-07-31 23:46:35.032798639 +0000 UTC m=+18.212255367 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs") pod "network-metrics-daemon-tn7kj" (UID: "75c8e3ed-6587-4ef9-b3bb-562bf74fec5b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:27.033222 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:27.032840    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret podName:6c1ae62a-0b92-4205-a4bb-5d266930b743 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:35.032829653 +0000 UTC m=+18.212286381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret") pod "global-pull-secret-syncer-4k4dl" (UID: "6c1ae62a-0b92-4205-a4bb-5d266930b743") : object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:27.133752 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:27.133697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87ptm\" (UniqueName: \"kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm\") pod \"network-check-target-frb4r\" (UID: \"1cc3f661-c981-4802-964e-632fab8107c6\") " pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:27.134055 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:27.133964    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 23:46:27.134055 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:27.134025    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 23:46:27.134055 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:27.134045    2581 projected.go:196] Error preparing data for projected volume kube-api-access-87ptm for pod openshift-network-diagnostics/network-check-target-frb4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:27.134253 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:27.134130    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm podName:1cc3f661-c981-4802-964e-632fab8107c6 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:35.134102551 +0000 UTC m=+18.313559296 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-87ptm" (UniqueName: "kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm") pod "network-check-target-frb4r" (UID: "1cc3f661-c981-4802-964e-632fab8107c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:27.384173 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:27.384070    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:27.384363 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:27.384206    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:27.384845 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:27.384680    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:27.384845 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:27.384793    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:28.382592 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:28.382551    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:28.383083 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:28.382710    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:29.382758 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:29.382671    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:29.383312 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:29.382814    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:29.383312 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:29.382841    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:29.383312 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:29.382976    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:30.383153 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:30.383115    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:30.383531 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:30.383238    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:31.382529 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:31.382481    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:31.382529 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:31.382529    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:31.382719 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:31.382627    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:31.382719 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:31.382705    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:32.382690 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:32.382659    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:32.383184 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:32.382815    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:33.382438 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:33.382402    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:33.382602 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:33.382537    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:33.382602 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:33.382584    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:33.382725 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:33.382706    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:34.382537 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:34.382502    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:34.382757 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:34.382631    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:35.093912 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:35.093868    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret\") pod \"global-pull-secret-syncer-4k4dl\" (UID: \"6c1ae62a-0b92-4205-a4bb-5d266930b743\") " pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:35.094124 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:35.093938    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs\") pod \"network-metrics-daemon-tn7kj\" (UID: \"75c8e3ed-6587-4ef9-b3bb-562bf74fec5b\") " pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:35.094124 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:35.094048    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:35.094124 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:35.094068    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:35.094124 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:35.094117    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret podName:6c1ae62a-0b92-4205-a4bb-5d266930b743 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:51.094098846 +0000 UTC m=+34.273555572 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret") pod "global-pull-secret-syncer-4k4dl" (UID: "6c1ae62a-0b92-4205-a4bb-5d266930b743") : object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:35.094305 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:35.094135    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs podName:75c8e3ed-6587-4ef9-b3bb-562bf74fec5b nodeName:}" failed. No retries permitted until 2026-07-31 23:46:51.094126833 +0000 UTC m=+34.273583556 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs") pod "network-metrics-daemon-tn7kj" (UID: "75c8e3ed-6587-4ef9-b3bb-562bf74fec5b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:35.194464 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:35.194423    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87ptm\" (UniqueName: \"kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm\") pod \"network-check-target-frb4r\" (UID: \"1cc3f661-c981-4802-964e-632fab8107c6\") " pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:35.194608 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:35.194593    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 23:46:35.194664 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:35.194618    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 23:46:35.194664 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:35.194628    2581 projected.go:196] Error preparing data for projected volume kube-api-access-87ptm for pod openshift-network-diagnostics/network-check-target-frb4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:35.194726 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:35.194683    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm podName:1cc3f661-c981-4802-964e-632fab8107c6 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:51.194665208 +0000 UTC m=+34.374121931 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-87ptm" (UniqueName: "kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm") pod "network-check-target-frb4r" (UID: "1cc3f661-c981-4802-964e-632fab8107c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:35.382825 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:35.382745    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:35.383197 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:35.382883    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:35.383197 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:35.382928    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:35.383197 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:35.383033    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:36.382259 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:36.382224    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:36.382516 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:36.382361    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:37.382837 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:37.382799    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:37.383313 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:37.382928    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:37.383313 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:37.382946    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:37.383313 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:37.383046    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:38.383200 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:38.383159    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:38.383619 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:38.383293    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:39.382274 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:39.382237    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:39.382439 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:39.382375    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:39.382439 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:39.382414    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:39.382613 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:39.382511    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:39.539850 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:39.539610    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dpmjk" event={"ID":"d8685c5a-294f-4e78-9504-085286bd468a","Type":"ContainerStarted","Data":"336835a4be254234db90893e98317dcd028d670cc6e2644a7724994e44eab03a"}
Jul 31 23:46:39.541674 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:39.541607    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zqrmw" event={"ID":"bb83a5d0-9c9a-4d84-8562-560971cad828","Type":"ContainerStarted","Data":"f072a48d74e3255bfc491ec17e4dcb3fb6065b4d230116374388dcb8914b83fa"}
Jul 31 23:46:39.557773 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:39.557710    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dpmjk" podStartSLOduration=2.5552522829999997 podStartE2EDuration="21.557684004s" totalImagesPullingTime="19.002431721s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:39.556471712 +0000 UTC m=+22.735928529" watchObservedRunningTime="2026-07-31 23:46:39.557684004 +0000 UTC m=+22.737140750"
Jul 31 23:46:39.645296 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:39.645266    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zqrmw"
Jul 31 23:46:39.645840 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:39.645822    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zqrmw"
Jul 31 23:46:39.662212 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:39.662150    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zqrmw" podStartSLOduration=10.528538414 podStartE2EDuration="21.662127471s" totalImagesPullingTime="11.133589057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:39.573108459 +0000 UTC m=+22.752565205" watchObservedRunningTime="2026-07-31 23:46:39.662127471 +0000 UTC m=+22.841584216"
Jul 31 23:46:40.383269 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.383057    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:40.383449 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:40.383376    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:40.544926 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.544885    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf" event={"ID":"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6","Type":"ContainerStarted","Data":"0d9d1ef4d7fecbc4d127aa24725daa2ed41817813c1f4cc5a814815eafff69c8"}
Jul 31 23:46:40.546276 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.546252    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-twdx5" event={"ID":"f85fb808-2a50-499e-a3a8-3aa1c33f69a2","Type":"ContainerStarted","Data":"a0b22d30429af23b02cdde97681893269a7873c1235bb689b84176a190e1ee3f"}
Jul 31 23:46:40.549296 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.549257    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw" event={"ID":"4869dda6-8fe5-4c60-abc7-f16ce429b2d4","Type":"ContainerStarted","Data":"c9232e297cc4f8be2c32aa7ec2a8b3fa4755195686407184e99dd98d98de3477"}
Jul 31 23:46:40.549431 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.549357    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw" event={"ID":"4869dda6-8fe5-4c60-abc7-f16ce429b2d4","Type":"ContainerStarted","Data":"75d227c95b955a3bbe1aa12652c8f079bcf504ee8372ab28362076adb59de23b"}
Jul 31 23:46:40.549431 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.549395    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw" event={"ID":"4869dda6-8fe5-4c60-abc7-f16ce429b2d4","Type":"ContainerStarted","Data":"d1afc6ed9af51901c42a5cf02c421ad9b0b5625938b35ccbc8fe3a99c63289cf"}
Jul 31 23:46:40.549535 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.549433    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw" event={"ID":"4869dda6-8fe5-4c60-abc7-f16ce429b2d4","Type":"ContainerStarted","Data":"d9c95c4209af354011686cc056e74d33f0dab778ca6dc7fd91aa26239a9d0b91"}
Jul 31 23:46:40.549535 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.549472    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw" event={"ID":"4869dda6-8fe5-4c60-abc7-f16ce429b2d4","Type":"ContainerStarted","Data":"62e44732a5c655f88bc4a5164356e977626f207bd2e4f2a9d5a6ee4ca3304aae"}
Jul 31 23:46:40.549535 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.549503    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw" event={"ID":"4869dda6-8fe5-4c60-abc7-f16ce429b2d4","Type":"ContainerStarted","Data":"f7c7bfd5d43c22ca93f3e0d97484f30d83f973684b4338947dbf3115d8d8dae6"}
Jul 31 23:46:40.550701 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.550677    2581 generic.go:356] "Generic (PLEG): container finished" podID="9da928a7-a1b0-4b64-a8a0-0e007b337734" containerID="d8564da70a5c637d404c2062a9257cd21bbd5ba479d62c9424721ccff5eefa0b" exitCode=0
Jul 31 23:46:40.550820 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.550749    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6bv6" event={"ID":"9da928a7-a1b0-4b64-a8a0-0e007b337734","Type":"ContainerDied","Data":"d8564da70a5c637d404c2062a9257cd21bbd5ba479d62c9424721ccff5eefa0b"}
Jul 31 23:46:40.552258 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.552237    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d25h7" event={"ID":"98dd5ea5-e96e-406a-956c-200a1e0314d1","Type":"ContainerStarted","Data":"36c3a07a21978adfff768bfb199fc52888604de8146acf474b0c71020efeaea1"}
Jul 31 23:46:40.553606 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.553579    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hftd5" event={"ID":"5dd3564c-d0e5-444f-9417-4d23ea2951e1","Type":"ContainerStarted","Data":"1c334d8ca9bf1ff8b152142fde3ad9a343286fff87b973706ab1b629504ee43f"}
Jul 31 23:46:40.560920 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.560870    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-twdx5" podStartSLOduration=3.558338113 podStartE2EDuration="22.56085645s" totalImagesPullingTime="19.002518337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:40.560459029 +0000 UTC m=+23.739915775" watchObservedRunningTime="2026-07-31 23:46:40.56085645 +0000 UTC m=+23.740313193"
Jul 31 23:46:40.576803 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.576750    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hftd5" podStartSLOduration=3.52061548 podStartE2EDuration="22.5767341s" totalImagesPullingTime="19.05611862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:40.576365638 +0000 UTC m=+23.755822383" watchObservedRunningTime="2026-07-31 23:46:40.5767341 +0000 UTC m=+23.756190844"
Jul 31 23:46:40.590527 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.590476    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-d25h7" podStartSLOduration=3.586161901 podStartE2EDuration="22.590463471s" totalImagesPullingTime="19.00430157s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:40.59014637 +0000 UTC m=+23.769603114" watchObservedRunningTime="2026-07-31 23:46:40.590463471 +0000 UTC m=+23.769920214"
Jul 31 23:46:40.899789 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:40.899757    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 31 23:46:41.318405 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:41.318255    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-31T23:46:40.899781313Z","UUID":"1d5ee7e1-1b13-4da2-8632-b24a144489d9","Handler":null,"Name":"","Endpoint":""}
Jul 31 23:46:41.320066 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:41.320042    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 31 23:46:41.320217 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:41.320081    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 31 23:46:41.382870 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:41.382836    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:41.382870 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:41.382862    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:41.383200 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:41.382989    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:41.383200 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:41.383061    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:41.558345 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:41.558296    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c8qbc" event={"ID":"6c600206-7804-41f1-ae49-de6c40a4d2e0","Type":"ContainerStarted","Data":"db5a8a08fc54b49b120b142fd87146969a14fe25d9dc57f515bd7b4d3a51e2ce"}
Jul 31 23:46:41.560704 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:41.560670    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf" event={"ID":"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6","Type":"ContainerStarted","Data":"783e0d2184200ba4ac572d4360962b1bdaf49d5e6912c6193ed1e11be08aadec"}
Jul 31 23:46:41.561142 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:41.560973    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 31 23:46:41.575066 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:41.574941    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-c8qbc" podStartSLOduration=4.565959586 podStartE2EDuration="23.574922436s" totalImagesPullingTime="19.00896285s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:41.574703291 +0000 UTC m=+24.754160035" watchObservedRunningTime="2026-07-31 23:46:41.574922436 +0000 UTC m=+24.754379181"
Jul 31 23:46:42.383108 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:42.382874    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:42.383329 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:42.383219    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:42.565827 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:42.565739    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf" event={"ID":"6ba5f015-bfbc-45c6-9e7f-255c700ecdb6","Type":"ContainerStarted","Data":"4e510c75d9a6b84b768010ab26b11d084aec543ff6b6a525f8bcf9025d6170a7"}
Jul 31 23:46:42.569651 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:42.569617    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw" event={"ID":"4869dda6-8fe5-4c60-abc7-f16ce429b2d4","Type":"ContainerStarted","Data":"edc6bf944b3245bc38deeb9cdf9d086721b4fa068934bc1a10333a500ab56f6f"}
Jul 31 23:46:42.582667 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:42.582613    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p44lf" podStartSLOduration=3.170258459 podStartE2EDuration="24.582595636s" totalImagesPullingTime="21.412337177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:42.5820239 +0000 UTC m=+25.761480643" watchObservedRunningTime="2026-07-31 23:46:42.582595636 +0000 UTC m=+25.762052380"
Jul 31 23:46:43.383275 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:43.383241    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:43.383488 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:43.383375    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:43.383488 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:43.383426    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:43.383613 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:43.383523    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:44.382823 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:44.382791    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:44.383347 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:44.382930    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:45.382758 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:45.382561    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:45.382935 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:45.382561    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:45.382935 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:45.382855    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:45.382935 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:45.382902    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:45.583407 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:45.583370    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw" event={"ID":"4869dda6-8fe5-4c60-abc7-f16ce429b2d4","Type":"ContainerStarted","Data":"58a0c991639aaee00027e0a2aa4262fc87fb9fa155082cbb4a6f0f620a7f2fb2"}
Jul 31 23:46:45.583692 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:45.583653    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:45.583830 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:45.583739    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:45.583830 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:45.583791    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:45.585348 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:45.585324    2581 generic.go:356] "Generic (PLEG): container finished" podID="9da928a7-a1b0-4b64-a8a0-0e007b337734" containerID="84e30842247727f35c3de222e385b609f32bdfd985c17252e8f1bf8f06dde01e" exitCode=0
Jul 31 23:46:45.585466 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:45.585368    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6bv6" event={"ID":"9da928a7-a1b0-4b64-a8a0-0e007b337734","Type":"ContainerDied","Data":"84e30842247727f35c3de222e385b609f32bdfd985c17252e8f1bf8f06dde01e"}
Jul 31 23:46:45.599525 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:45.599478    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:45.599668 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:45.599580    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:46:45.612471 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:45.612427    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw" podStartSLOduration=8.537624723 podStartE2EDuration="27.612413161s" totalImagesPullingTime="19.074788438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:45.610871578 +0000 UTC m=+28.790328323" watchObservedRunningTime="2026-07-31 23:46:45.612413161 +0000 UTC m=+28.791869904"
Jul 31 23:46:46.382444 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:46.382270    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:46.382633 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:46.382558    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:46.547785 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:46.547752    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4k4dl"]
Jul 31 23:46:46.548245 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:46.547911    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:46.548245 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:46.548049    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:46.548562 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:46.548537    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-frb4r"]
Jul 31 23:46:46.548691 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:46.548638    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:46.548764 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:46.548732    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:46.549186 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:46.549159    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tn7kj"]
Jul 31 23:46:46.590478 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:46.590390    2581 generic.go:356] "Generic (PLEG): container finished" podID="9da928a7-a1b0-4b64-a8a0-0e007b337734" containerID="349a15edca91af586ddb24b64725a08db2fa16a28efaf14b731e8d1f20bb428a" exitCode=0
Jul 31 23:46:46.590478 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:46.590467    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:46.590478 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:46.590466    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6bv6" event={"ID":"9da928a7-a1b0-4b64-a8a0-0e007b337734","Type":"ContainerDied","Data":"349a15edca91af586ddb24b64725a08db2fa16a28efaf14b731e8d1f20bb428a"}
Jul 31 23:46:46.590819 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:46.590794    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:47.903187 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:47.903099    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zqrmw"
Jul 31 23:46:47.903625 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:47.903253    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 31 23:46:47.903747 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:47.903724    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zqrmw"
Jul 31 23:46:48.382519 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:48.382434    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:48.382519 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:48.382469    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:48.382755 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:48.382540    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:48.382755 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:48.382434    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:48.382755 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:48.382648    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:48.382755 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:48.382744    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:48.597015 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:48.596984    2581 generic.go:356] "Generic (PLEG): container finished" podID="9da928a7-a1b0-4b64-a8a0-0e007b337734" containerID="a4c3f61feab75e017da6fed396859f896094eca174d8f2e11547339ca6658ea8" exitCode=0
Jul 31 23:46:48.597159 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:48.597062    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6bv6" event={"ID":"9da928a7-a1b0-4b64-a8a0-0e007b337734","Type":"ContainerDied","Data":"a4c3f61feab75e017da6fed396859f896094eca174d8f2e11547339ca6658ea8"}
Jul 31 23:46:50.382790 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:50.382758    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:50.382790 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:50.382776    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:50.382790 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:50.382799    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:50.383789 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:50.382897    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:50.383789 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:50.382991    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:50.383789 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:50.383058    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:51.118124 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:51.118090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs\") pod \"network-metrics-daemon-tn7kj\" (UID: \"75c8e3ed-6587-4ef9-b3bb-562bf74fec5b\") " pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:51.118333 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:51.118156    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret\") pod \"global-pull-secret-syncer-4k4dl\" (UID: \"6c1ae62a-0b92-4205-a4bb-5d266930b743\") " pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:51.118333 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:51.118252    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:51.118333 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:51.118312    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret podName:6c1ae62a-0b92-4205-a4bb-5d266930b743 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:23.118295107 +0000 UTC m=+66.297751832 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret") pod "global-pull-secret-syncer-4k4dl" (UID: "6c1ae62a-0b92-4205-a4bb-5d266930b743") : object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:51.118333 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:51.118253    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:51.118554 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:51.118398    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs podName:75c8e3ed-6587-4ef9-b3bb-562bf74fec5b nodeName:}" failed. No retries permitted until 2026-07-31 23:47:23.118380365 +0000 UTC m=+66.297837098 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs") pod "network-metrics-daemon-tn7kj" (UID: "75c8e3ed-6587-4ef9-b3bb-562bf74fec5b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:51.219401 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:51.219360    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87ptm\" (UniqueName: \"kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm\") pod \"network-check-target-frb4r\" (UID: \"1cc3f661-c981-4802-964e-632fab8107c6\") " pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:51.219604 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:51.219559    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 23:46:51.219604 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:51.219585    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 23:46:51.219604 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:51.219598    2581 projected.go:196] Error preparing data for projected volume kube-api-access-87ptm for pod openshift-network-diagnostics/network-check-target-frb4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:51.219761 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:51.219664    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm podName:1cc3f661-c981-4802-964e-632fab8107c6 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:23.219644161 +0000 UTC m=+66.399100904 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-87ptm" (UniqueName: "kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm") pod "network-check-target-frb4r" (UID: "1cc3f661-c981-4802-964e-632fab8107c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:52.383061 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:52.383018    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:52.383061 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:52.383056    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:52.383562 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:52.383056    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:52.383562 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:52.383161    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tn7kj" podUID="75c8e3ed-6587-4ef9-b3bb-562bf74fec5b"
Jul 31 23:46:52.383562 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:52.383238    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frb4r" podUID="1cc3f661-c981-4802-964e-632fab8107c6"
Jul 31 23:46:52.383562 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:52.383334    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k4dl" podUID="6c1ae62a-0b92-4205-a4bb-5d266930b743"
Jul 31 23:46:53.121877 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.121801    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-141.ec2.internal" event="NodeReady"
Jul 31 23:46:53.122078 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.121951    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 31 23:46:53.168618 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.168541    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-67cs9"]
Jul 31 23:46:53.172498 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.172473    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-574d844ff-fdscc"]
Jul 31 23:46:53.172635 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.172604    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-67cs9"
Jul 31 23:46:53.175357 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.175324    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 31 23:46:53.175357 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.175324    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 31 23:46:53.175529 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.175366    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 31 23:46:53.175529 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.175493    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-m6h4p\""
Jul 31 23:46:53.175898 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.175883    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-x8242"]
Jul 31 23:46:53.176020 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.176008    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.178521 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.178500    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 31 23:46:53.178663 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.178649    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fxfp4\""
Jul 31 23:46:53.178935 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.178912    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 31 23:46:53.179029 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.179013    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 31 23:46:53.179177 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.179152    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-67cs9"]
Jul 31 23:46:53.179282 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.179269    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x8242"
Jul 31 23:46:53.184538 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.183307    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 31 23:46:53.184538 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.183781    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 31 23:46:53.184538 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.184336    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rs996\""
Jul 31 23:46:53.188940 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.186720    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x8242"]
Jul 31 23:46:53.188940 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.187948    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-574d844ff-fdscc"]
Jul 31 23:46:53.191829 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.191440    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 31 23:46:53.336452 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.336409    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sprn9\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-kube-api-access-sprn9\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.336624 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.336468    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72d94379-6a47-4f1a-af3b-25e254074711-cert\") pod \"ingress-canary-67cs9\" (UID: \"72d94379-6a47-4f1a-af3b-25e254074711\") " pod="openshift-ingress-canary/ingress-canary-67cs9"
Jul 31 23:46:53.336624 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.336499    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:46:53.336624 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.336546    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-certificates\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.336624 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.336564    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-bound-sa-token\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.336624 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.336615    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqfm2\" (UniqueName: \"kubernetes.io/projected/72d94379-6a47-4f1a-af3b-25e254074711-kube-api-access-wqfm2\") pod \"ingress-canary-67cs9\" (UID: \"72d94379-6a47-4f1a-af3b-25e254074711\") " pod="openshift-ingress-canary/ingress-canary-67cs9"
Jul 31 23:46:53.336775 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.336642    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8a8e52b-93a1-42df-acca-45fe1b85c275-image-registry-private-configuration\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.336775 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.336675    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4c461a2c-8de2-4813-9cfd-45366645a56e-config-volume\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:46:53.336775 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.336700    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.336775 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.336716    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8a8e52b-93a1-42df-acca-45fe1b85c275-installation-pull-secrets\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.336775 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.336735    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8a8e52b-93a1-42df-acca-45fe1b85c275-trusted-ca\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.336775 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.336750    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c461a2c-8de2-4813-9cfd-45366645a56e-tmp-dir\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:46:53.336775 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.336774    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7x75\" (UniqueName: \"kubernetes.io/projected/4c461a2c-8de2-4813-9cfd-45366645a56e-kube-api-access-d7x75\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:46:53.336994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.336792    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8a8e52b-93a1-42df-acca-45fe1b85c275-ca-trust-extracted\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.437436 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.437397    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sprn9\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-kube-api-access-sprn9\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.437855 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.437451    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72d94379-6a47-4f1a-af3b-25e254074711-cert\") pod \"ingress-canary-67cs9\" (UID: \"72d94379-6a47-4f1a-af3b-25e254074711\") " pod="openshift-ingress-canary/ingress-canary-67cs9"
Jul 31 23:46:53.437855 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.437468    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:46:53.437855 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.437490    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-certificates\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.437855 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:53.437578    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 23:46:53.437855 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.437607    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-bound-sa-token\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.437855 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:53.437647    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls podName:4c461a2c-8de2-4813-9cfd-45366645a56e nodeName:}" failed. No retries permitted until 2026-07-31 23:46:53.937631078 +0000 UTC m=+37.117087805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls") pod "dns-default-x8242" (UID: "4c461a2c-8de2-4813-9cfd-45366645a56e") : secret "dns-default-metrics-tls" not found
Jul 31 23:46:53.437855 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.437674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqfm2\" (UniqueName: \"kubernetes.io/projected/72d94379-6a47-4f1a-af3b-25e254074711-kube-api-access-wqfm2\") pod \"ingress-canary-67cs9\" (UID: \"72d94379-6a47-4f1a-af3b-25e254074711\") " pod="openshift-ingress-canary/ingress-canary-67cs9"
Jul 31 23:46:53.437855 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.437693    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8a8e52b-93a1-42df-acca-45fe1b85c275-image-registry-private-configuration\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.437855 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.437717    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4c461a2c-8de2-4813-9cfd-45366645a56e-config-volume\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:46:53.437855 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.437738    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.437855 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.437762    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8a8e52b-93a1-42df-acca-45fe1b85c275-installation-pull-secrets\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.437855 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.437791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8a8e52b-93a1-42df-acca-45fe1b85c275-trusted-ca\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.437855 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.437814    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c461a2c-8de2-4813-9cfd-45366645a56e-tmp-dir\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:46:53.437855 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.437851    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7x75\" (UniqueName: \"kubernetes.io/projected/4c461a2c-8de2-4813-9cfd-45366645a56e-kube-api-access-d7x75\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:46:53.438559 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.437880    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8a8e52b-93a1-42df-acca-45fe1b85c275-ca-trust-extracted\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.438559 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:53.437997    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 23:46:53.438559 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:53.438011    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574d844ff-fdscc: secret "image-registry-tls" not found
Jul 31 23:46:53.438559 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:53.438075    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls podName:e8a8e52b-93a1-42df-acca-45fe1b85c275 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:53.938057456 +0000 UTC m=+37.117514197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls") pod "image-registry-574d844ff-fdscc" (UID: "e8a8e52b-93a1-42df-acca-45fe1b85c275") : secret "image-registry-tls" not found
Jul 31 23:46:53.438559 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.438207    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-certificates\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.438559 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.438303    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c461a2c-8de2-4813-9cfd-45366645a56e-tmp-dir\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:46:53.438559 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.438340    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4c461a2c-8de2-4813-9cfd-45366645a56e-config-volume\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:46:53.438559 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.438501    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8a8e52b-93a1-42df-acca-45fe1b85c275-ca-trust-extracted\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.438893 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.438875    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8a8e52b-93a1-42df-acca-45fe1b85c275-trusted-ca\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.441835 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.441804    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8a8e52b-93a1-42df-acca-45fe1b85c275-installation-pull-secrets\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.442000 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.441843    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8a8e52b-93a1-42df-acca-45fe1b85c275-image-registry-private-configuration\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.442000 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.441900    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72d94379-6a47-4f1a-af3b-25e254074711-cert\") pod \"ingress-canary-67cs9\" (UID: \"72d94379-6a47-4f1a-af3b-25e254074711\") " pod="openshift-ingress-canary/ingress-canary-67cs9"
Jul 31 23:46:53.446678 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.446648    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqfm2\" (UniqueName: \"kubernetes.io/projected/72d94379-6a47-4f1a-af3b-25e254074711-kube-api-access-wqfm2\") pod \"ingress-canary-67cs9\" (UID: \"72d94379-6a47-4f1a-af3b-25e254074711\") " pod="openshift-ingress-canary/ingress-canary-67cs9"
Jul 31 23:46:53.446805 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.446771    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-bound-sa-token\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.447050 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.447031    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sprn9\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-kube-api-access-sprn9\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.447370 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.447349    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7x75\" (UniqueName: \"kubernetes.io/projected/4c461a2c-8de2-4813-9cfd-45366645a56e-kube-api-access-d7x75\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:46:53.485634 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.485577    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-67cs9"
Jul 31 23:46:53.612330 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.612292    2581 generic.go:356] "Generic (PLEG): container finished" podID="9da928a7-a1b0-4b64-a8a0-0e007b337734" containerID="a468431c84407582a0a198b53e0d5c3ffa43ac5fd4c6d9a1e3a4c24c47727455" exitCode=0
Jul 31 23:46:53.612480 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.612349    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6bv6" event={"ID":"9da928a7-a1b0-4b64-a8a0-0e007b337734","Type":"ContainerDied","Data":"a468431c84407582a0a198b53e0d5c3ffa43ac5fd4c6d9a1e3a4c24c47727455"}
Jul 31 23:46:53.625403 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.625376    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-67cs9"]
Jul 31 23:46:53.630754 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:46:53.630723    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72d94379_6a47_4f1a_af3b_25e254074711.slice/crio-13b3412beb22b276e4385cf156059c1612ee0c8ff7623003ab945ec497fa7bf6 WatchSource:0}: Error finding container 13b3412beb22b276e4385cf156059c1612ee0c8ff7623003ab945ec497fa7bf6: Status 404 returned error can't find the container with id 13b3412beb22b276e4385cf156059c1612ee0c8ff7623003ab945ec497fa7bf6
Jul 31 23:46:53.942646 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.942604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:53.942867 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:53.942681    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:46:53.942867 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:53.942771    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 23:46:53.942867 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:53.942793    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574d844ff-fdscc: secret "image-registry-tls" not found
Jul 31 23:46:53.942867 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:53.942846    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls podName:e8a8e52b-93a1-42df-acca-45fe1b85c275 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:54.942830999 +0000 UTC m=+38.122287721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls") pod "image-registry-574d844ff-fdscc" (UID: "e8a8e52b-93a1-42df-acca-45fe1b85c275") : secret "image-registry-tls" not found
Jul 31 23:46:53.942867 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:53.942780    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 23:46:53.943101 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:53.942925    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls podName:4c461a2c-8de2-4813-9cfd-45366645a56e nodeName:}" failed. No retries permitted until 2026-07-31 23:46:54.942906826 +0000 UTC m=+38.122363553 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls") pod "dns-default-x8242" (UID: "4c461a2c-8de2-4813-9cfd-45366645a56e") : secret "dns-default-metrics-tls" not found
Jul 31 23:46:54.383035 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:54.382781    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:46:54.383232 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:54.382790    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:46:54.383232 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:54.382819    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:46:54.385997 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:54.385950    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 31 23:46:54.386681 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:54.386657    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 31 23:46:54.386681 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:54.386679    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5t5tm\""
Jul 31 23:46:54.386975 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:54.386945    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 31 23:46:54.387068 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:54.387034    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v7qx8\""
Jul 31 23:46:54.387128 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:54.387036    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 31 23:46:54.616797 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:54.616758    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-67cs9" event={"ID":"72d94379-6a47-4f1a-af3b-25e254074711","Type":"ContainerStarted","Data":"13b3412beb22b276e4385cf156059c1612ee0c8ff7623003ab945ec497fa7bf6"}
Jul 31 23:46:54.620464 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:54.620431    2581 generic.go:356] "Generic (PLEG): container finished" podID="9da928a7-a1b0-4b64-a8a0-0e007b337734" containerID="78f5d0c145235843141b3c416005bcdfbfb5f4109ec4b1688c6b81a3b55c1a9d" exitCode=0
Jul 31 23:46:54.620643 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:54.620476    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6bv6" event={"ID":"9da928a7-a1b0-4b64-a8a0-0e007b337734","Type":"ContainerDied","Data":"78f5d0c145235843141b3c416005bcdfbfb5f4109ec4b1688c6b81a3b55c1a9d"}
Jul 31 23:46:54.952159 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:54.952084    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:46:54.952159 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:54.952146    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:54.952314 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:54.952240    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 23:46:54.952314 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:54.952257    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 23:46:54.952314 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:54.952269    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574d844ff-fdscc: secret "image-registry-tls" not found
Jul 31 23:46:54.952314 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:54.952308    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls podName:4c461a2c-8de2-4813-9cfd-45366645a56e nodeName:}" failed. No retries permitted until 2026-07-31 23:46:56.952291516 +0000 UTC m=+40.131748255 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls") pod "dns-default-x8242" (UID: "4c461a2c-8de2-4813-9cfd-45366645a56e") : secret "dns-default-metrics-tls" not found
Jul 31 23:46:54.952439 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:54.952323    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls podName:e8a8e52b-93a1-42df-acca-45fe1b85c275 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:56.952317095 +0000 UTC m=+40.131773817 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls") pod "image-registry-574d844ff-fdscc" (UID: "e8a8e52b-93a1-42df-acca-45fe1b85c275") : secret "image-registry-tls" not found
Jul 31 23:46:55.627068 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:55.627027    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6bv6" event={"ID":"9da928a7-a1b0-4b64-a8a0-0e007b337734","Type":"ContainerStarted","Data":"159d875902216163e22047d3e1ba025a8acbb9dd0fe5224b61a5ddc46befddcc"}
Jul 31 23:46:55.650615 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:55.650564    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-q6bv6" podStartSLOduration=8.937246495 podStartE2EDuration="37.65054722s" totalImagesPullingTime="28.713300725s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:55.648998799 +0000 UTC m=+38.828455543" watchObservedRunningTime="2026-07-31 23:46:55.65054722 +0000 UTC m=+38.830003963"
Jul 31 23:46:56.631174 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:56.631141    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-67cs9" event={"ID":"72d94379-6a47-4f1a-af3b-25e254074711","Type":"ContainerStarted","Data":"cc4e1e7859753ebd8207517806a78a8e807973cd33b7ee05dbbf1fea68c24c02"}
Jul 31 23:46:56.649964 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:56.649915    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-67cs9" podStartSLOduration=1.401643453 podStartE2EDuration="3.649899031s" totalImagesPullingTime="2.248255578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:46:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:56.649172996 +0000 UTC m=+39.828629740" watchObservedRunningTime="2026-07-31 23:46:56.649899031 +0000 UTC m=+39.829355753"
Jul 31 23:46:56.970488 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:56.970389    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:46:56.970488 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:56.970446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:46:56.970683 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:56.970544    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 23:46:56.970683 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:56.970556    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 23:46:56.970683 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:56.970568    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574d844ff-fdscc: secret "image-registry-tls" not found
Jul 31 23:46:56.970683 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:56.970617    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls podName:4c461a2c-8de2-4813-9cfd-45366645a56e nodeName:}" failed. No retries permitted until 2026-07-31 23:47:00.970601805 +0000 UTC m=+44.150058527 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls") pod "dns-default-x8242" (UID: "4c461a2c-8de2-4813-9cfd-45366645a56e") : secret "dns-default-metrics-tls" not found
Jul 31 23:46:56.970683 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:56.970633    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls podName:e8a8e52b-93a1-42df-acca-45fe1b85c275 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:00.970627104 +0000 UTC m=+44.150083826 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls") pod "image-registry-574d844ff-fdscc" (UID: "e8a8e52b-93a1-42df-acca-45fe1b85c275") : secret "image-registry-tls" not found
Jul 31 23:46:59.585290 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.585256    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dcbx4"]
Jul 31 23:46:59.615442 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.615411    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dcbx4"]
Jul 31 23:46:59.615636 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.615552    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:46:59.618235 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.618214    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 31 23:46:59.618235 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.618223    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 31 23:46:59.618235 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.618221    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fst58\""
Jul 31 23:46:59.619182 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.619164    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 31 23:46:59.619270 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.619166    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 31 23:46:59.790637 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.790600    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-crio-socket\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:46:59.790637 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.790641    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:46:59.790880 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.790668    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq89j\" (UniqueName: \"kubernetes.io/projected/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-api-access-dq89j\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:46:59.790880 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.790769    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:46:59.790880 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.790837    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-data-volume\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:46:59.892174 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.892133    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-crio-socket\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:46:59.892337 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.892182    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:46:59.892337 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.892217    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq89j\" (UniqueName: \"kubernetes.io/projected/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-api-access-dq89j\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:46:59.892337 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.892254    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:46:59.892337 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:59.892334    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 23:46:59.892546 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.892334    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-crio-socket\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:46:59.892546 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:59.892345    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 23:46:59.892546 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.892392    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-data-volume\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:46:59.892546 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:59.892400    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-rbac-proxy-cm podName:4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f nodeName:}" failed. No retries permitted until 2026-07-31 23:47:00.392381068 +0000 UTC m=+43.571837806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dcbx4" (UID: "4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f") : configmap "kube-rbac-proxy" not found
Jul 31 23:46:59.892546 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:46:59.892467    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls podName:4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f nodeName:}" failed. No retries permitted until 2026-07-31 23:47:00.392454597 +0000 UTC m=+43.571911324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dcbx4" (UID: "4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f") : secret "insights-runtime-extractor-tls" not found
Jul 31 23:46:59.892724 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.892684    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-data-volume\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:46:59.913258 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:46:59.913228    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq89j\" (UniqueName: \"kubernetes.io/projected/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-api-access-dq89j\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:47:00.397077 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:00.397041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:47:00.397247 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:00.397169    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 23:47:00.397247 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:00.397170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:47:00.397247 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:00.397241    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-rbac-proxy-cm podName:4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f nodeName:}" failed. No retries permitted until 2026-07-31 23:47:01.397220555 +0000 UTC m=+44.576677283 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dcbx4" (UID: "4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f") : configmap "kube-rbac-proxy" not found
Jul 31 23:47:00.397354 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:00.397252    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 23:47:00.397354 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:00.397307    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls podName:4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f nodeName:}" failed. No retries permitted until 2026-07-31 23:47:01.397295266 +0000 UTC m=+44.576751993 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dcbx4" (UID: "4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f") : secret "insights-runtime-extractor-tls" not found
Jul 31 23:47:01.001136 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:01.001092    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:47:01.001607 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:01.001204    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:47:01.001607 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:01.001235    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 23:47:01.001607 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:01.001259    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574d844ff-fdscc: secret "image-registry-tls" not found
Jul 31 23:47:01.001607 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:01.001313    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls podName:e8a8e52b-93a1-42df-acca-45fe1b85c275 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:09.00129822 +0000 UTC m=+52.180754946 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls") pod "image-registry-574d844ff-fdscc" (UID: "e8a8e52b-93a1-42df-acca-45fe1b85c275") : secret "image-registry-tls" not found
Jul 31 23:47:01.001607 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:01.001320    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 23:47:01.001607 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:01.001369    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls podName:4c461a2c-8de2-4813-9cfd-45366645a56e nodeName:}" failed. No retries permitted until 2026-07-31 23:47:09.001355298 +0000 UTC m=+52.180812019 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls") pod "dns-default-x8242" (UID: "4c461a2c-8de2-4813-9cfd-45366645a56e") : secret "dns-default-metrics-tls" not found
Jul 31 23:47:01.404509 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:01.404471    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:47:01.404683 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:01.404587    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 23:47:01.404683 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:01.404634    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:47:01.404683 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:01.404647    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-rbac-proxy-cm podName:4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f nodeName:}" failed. No retries permitted until 2026-07-31 23:47:03.404633583 +0000 UTC m=+46.584090305 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dcbx4" (UID: "4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f") : configmap "kube-rbac-proxy" not found
Jul 31 23:47:01.404792 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:01.404703    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 23:47:01.404792 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:01.404743    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls podName:4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f nodeName:}" failed. No retries permitted until 2026-07-31 23:47:03.404732514 +0000 UTC m=+46.584189236 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dcbx4" (UID: "4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f") : secret "insights-runtime-extractor-tls" not found
Jul 31 23:47:01.643942 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:01.643912    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-twdx5_f85fb808-2a50-499e-a3a8-3aa1c33f69a2/dns-node-resolver/0.log"
Jul 31 23:47:02.648713 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:02.648683    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d25h7_98dd5ea5-e96e-406a-956c-200a1e0314d1/node-ca/0.log"
Jul 31 23:47:03.246120 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:03.246084    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-67cs9_72d94379-6a47-4f1a-af3b-25e254074711/serve-healthcheck-canary/0.log"
Jul 31 23:47:03.419230 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:03.419191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:47:03.419381 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:03.419243    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:47:03.419381 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:03.419321    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 31 23:47:03.419381 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:03.419335    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 23:47:03.419381 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:03.419376    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-rbac-proxy-cm podName:4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f nodeName:}" failed. No retries permitted until 2026-07-31 23:47:07.419363057 +0000 UTC m=+50.598819780 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dcbx4" (UID: "4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f") : configmap "kube-rbac-proxy" not found
Jul 31 23:47:03.419521 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:03.419389    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls podName:4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f nodeName:}" failed. No retries permitted until 2026-07-31 23:47:07.419383349 +0000 UTC m=+50.598840071 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dcbx4" (UID: "4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f") : secret "insights-runtime-extractor-tls" not found
Jul 31 23:47:07.448266 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:07.448221    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:47:07.448266 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:07.448284    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:47:07.448721 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:07.448410    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 23:47:07.448721 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:07.448482    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls podName:4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f nodeName:}" failed. No retries permitted until 2026-07-31 23:47:15.448465814 +0000 UTC m=+58.627922536 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dcbx4" (UID: "4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f") : secret "insights-runtime-extractor-tls" not found
Jul 31 23:47:07.448831 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:07.448813    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:47:09.059177 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:09.059137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls\") pod \"image-registry-574d844ff-fdscc\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") " pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:47:09.059565 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:09.059207    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:47:09.059565 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:09.059296    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 31 23:47:09.059565 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:09.059323    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-574d844ff-fdscc: secret "image-registry-tls" not found
Jul 31 23:47:09.059565 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:09.059385    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls podName:e8a8e52b-93a1-42df-acca-45fe1b85c275 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:25.05936976 +0000 UTC m=+68.238826485 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls") pod "image-registry-574d844ff-fdscc" (UID: "e8a8e52b-93a1-42df-acca-45fe1b85c275") : secret "image-registry-tls" not found
Jul 31 23:47:09.059565 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:09.059300    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 31 23:47:09.059565 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:09.059429    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls podName:4c461a2c-8de2-4813-9cfd-45366645a56e nodeName:}" failed. No retries permitted until 2026-07-31 23:47:25.059418188 +0000 UTC m=+68.238874909 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls") pod "dns-default-x8242" (UID: "4c461a2c-8de2-4813-9cfd-45366645a56e") : secret "dns-default-metrics-tls" not found
Jul 31 23:47:15.509377 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:15.509326    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:47:15.509911 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:15.509459    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 31 23:47:15.509911 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:15.509528    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls podName:4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f nodeName:}" failed. No retries permitted until 2026-07-31 23:47:31.509509059 +0000 UTC m=+74.688965783 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dcbx4" (UID: "4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f") : secret "insights-runtime-extractor-tls" not found
Jul 31 23:47:17.610208 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:17.610176    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5sqvw"
Jul 31 23:47:22.363970 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.363890    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-574d844ff-fdscc"]
Jul 31 23:47:22.364610 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:22.364541    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-574d844ff-fdscc" podUID="e8a8e52b-93a1-42df-acca-45fe1b85c275"
Jul 31 23:47:22.367189 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.367163    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5"]
Jul 31 23:47:22.369596 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.369581    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"]
Jul 31 23:47:22.369706 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.369688    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5"
Jul 31 23:47:22.371249 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.371229    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.372351 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.372329    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 31 23:47:22.372512 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.372496    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 31 23:47:22.372596 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.372500    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 31 23:47:22.372824 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.372809    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 31 23:47:22.374185 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.374165    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 31 23:47:22.374371 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.374166    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 31 23:47:22.374440 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.374176    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 31 23:47:22.375042 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.375017    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 31 23:47:22.378563 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.378538    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5"]
Jul 31 23:47:22.384028 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.384001    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"]
Jul 31 23:47:22.466134 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.466091    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr8gr\" (UniqueName: \"kubernetes.io/projected/26b85f59-9239-4d95-a3f2-47de1ec9648e-kube-api-access-pr8gr\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.466310 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.466155    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/436d6b79-58a5-4a66-9ce7-07f66df1c251-tmp\") pod \"klusterlet-addon-workmgr-84897dbdb8-dh7l5\" (UID: \"436d6b79-58a5-4a66-9ce7-07f66df1c251\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5"
Jul 31 23:47:22.466310 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.466171    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/26b85f59-9239-4d95-a3f2-47de1ec9648e-ca\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.466310 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.466208    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/26b85f59-9239-4d95-a3f2-47de1ec9648e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.466310 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.466304    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/26b85f59-9239-4d95-a3f2-47de1ec9648e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.466444 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.466339    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b85f59-9239-4d95-a3f2-47de1ec9648e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.466444 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.466359    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/26b85f59-9239-4d95-a3f2-47de1ec9648e-hub\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.466444 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.466376    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/436d6b79-58a5-4a66-9ce7-07f66df1c251-klusterlet-config\") pod \"klusterlet-addon-workmgr-84897dbdb8-dh7l5\" (UID: \"436d6b79-58a5-4a66-9ce7-07f66df1c251\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5"
Jul 31 23:47:22.466444 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.466399    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d59n7\" (UniqueName: \"kubernetes.io/projected/436d6b79-58a5-4a66-9ce7-07f66df1c251-kube-api-access-d59n7\") pod \"klusterlet-addon-workmgr-84897dbdb8-dh7l5\" (UID: \"436d6b79-58a5-4a66-9ce7-07f66df1c251\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5"
Jul 31 23:47:22.567691 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.567654    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/26b85f59-9239-4d95-a3f2-47de1ec9648e-hub\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.567691 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.567693    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/436d6b79-58a5-4a66-9ce7-07f66df1c251-klusterlet-config\") pod \"klusterlet-addon-workmgr-84897dbdb8-dh7l5\" (UID: \"436d6b79-58a5-4a66-9ce7-07f66df1c251\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5"
Jul 31 23:47:22.567946 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.567718    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d59n7\" (UniqueName: \"kubernetes.io/projected/436d6b79-58a5-4a66-9ce7-07f66df1c251-kube-api-access-d59n7\") pod \"klusterlet-addon-workmgr-84897dbdb8-dh7l5\" (UID: \"436d6b79-58a5-4a66-9ce7-07f66df1c251\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5"
Jul 31 23:47:22.567946 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.567774    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr8gr\" (UniqueName: \"kubernetes.io/projected/26b85f59-9239-4d95-a3f2-47de1ec9648e-kube-api-access-pr8gr\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.567946 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.567810    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/436d6b79-58a5-4a66-9ce7-07f66df1c251-tmp\") pod \"klusterlet-addon-workmgr-84897dbdb8-dh7l5\" (UID: \"436d6b79-58a5-4a66-9ce7-07f66df1c251\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5"
Jul 31 23:47:22.567946 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.567824    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/26b85f59-9239-4d95-a3f2-47de1ec9648e-ca\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.567946 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.567845    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/26b85f59-9239-4d95-a3f2-47de1ec9648e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.567946 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.567878    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/26b85f59-9239-4d95-a3f2-47de1ec9648e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.567946 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.567908    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b85f59-9239-4d95-a3f2-47de1ec9648e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.568416 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.568259    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/436d6b79-58a5-4a66-9ce7-07f66df1c251-tmp\") pod \"klusterlet-addon-workmgr-84897dbdb8-dh7l5\" (UID: \"436d6b79-58a5-4a66-9ce7-07f66df1c251\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5"
Jul 31 23:47:22.568665 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.568638    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b85f59-9239-4d95-a3f2-47de1ec9648e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.570529 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.570502    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/26b85f59-9239-4d95-a3f2-47de1ec9648e-hub\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.570629 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.570553    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/26b85f59-9239-4d95-a3f2-47de1ec9648e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.570629 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.570573    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/26b85f59-9239-4d95-a3f2-47de1ec9648e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.570897 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.570876    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/436d6b79-58a5-4a66-9ce7-07f66df1c251-klusterlet-config\") pod \"klusterlet-addon-workmgr-84897dbdb8-dh7l5\" (UID: \"436d6b79-58a5-4a66-9ce7-07f66df1c251\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5"
Jul 31 23:47:22.570933 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.570919    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/26b85f59-9239-4d95-a3f2-47de1ec9648e-ca\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.577005 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.576979    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d59n7\" (UniqueName: \"kubernetes.io/projected/436d6b79-58a5-4a66-9ce7-07f66df1c251-kube-api-access-d59n7\") pod \"klusterlet-addon-workmgr-84897dbdb8-dh7l5\" (UID: \"436d6b79-58a5-4a66-9ce7-07f66df1c251\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5"
Jul 31 23:47:22.577457 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.577413    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr8gr\" (UniqueName: \"kubernetes.io/projected/26b85f59-9239-4d95-a3f2-47de1ec9648e-kube-api-access-pr8gr\") pod \"cluster-proxy-proxy-agent-866b48d95-xbsqw\" (UID: \"26b85f59-9239-4d95-a3f2-47de1ec9648e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.680817 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.680786    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5"
Jul 31 23:47:22.690183 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.690146    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:47:22.699000 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.698972    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"
Jul 31 23:47:22.701187 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.701166    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:47:22.769458 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.769407    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-certificates\") pod \"e8a8e52b-93a1-42df-acca-45fe1b85c275\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") "
Jul 31 23:47:22.769625 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.769461    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sprn9\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-kube-api-access-sprn9\") pod \"e8a8e52b-93a1-42df-acca-45fe1b85c275\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") "
Jul 31 23:47:22.769625 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.769489    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8a8e52b-93a1-42df-acca-45fe1b85c275-ca-trust-extracted\") pod \"e8a8e52b-93a1-42df-acca-45fe1b85c275\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") "
Jul 31 23:47:22.769625 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.769548    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8a8e52b-93a1-42df-acca-45fe1b85c275-trusted-ca\") pod \"e8a8e52b-93a1-42df-acca-45fe1b85c275\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") "
Jul 31 23:47:22.769625 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.769588    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-bound-sa-token\") pod \"e8a8e52b-93a1-42df-acca-45fe1b85c275\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") "
Jul 31 23:47:22.769625 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.769616    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8a8e52b-93a1-42df-acca-45fe1b85c275-image-registry-private-configuration\") pod \"e8a8e52b-93a1-42df-acca-45fe1b85c275\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") "
Jul 31 23:47:22.769856 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.769639    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8a8e52b-93a1-42df-acca-45fe1b85c275-installation-pull-secrets\") pod \"e8a8e52b-93a1-42df-acca-45fe1b85c275\" (UID: \"e8a8e52b-93a1-42df-acca-45fe1b85c275\") "
Jul 31 23:47:22.770494 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.770427    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a8e52b-93a1-42df-acca-45fe1b85c275-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e8a8e52b-93a1-42df-acca-45fe1b85c275" (UID: "e8a8e52b-93a1-42df-acca-45fe1b85c275"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 23:47:22.770620 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.770563    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e8a8e52b-93a1-42df-acca-45fe1b85c275" (UID: "e8a8e52b-93a1-42df-acca-45fe1b85c275"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:47:22.771214 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.770942    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8a8e52b-93a1-42df-acca-45fe1b85c275-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e8a8e52b-93a1-42df-acca-45fe1b85c275" (UID: "e8a8e52b-93a1-42df-acca-45fe1b85c275"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:47:22.781221 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.781164    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-kube-api-access-sprn9" (OuterVolumeSpecName: "kube-api-access-sprn9") pod "e8a8e52b-93a1-42df-acca-45fe1b85c275" (UID: "e8a8e52b-93a1-42df-acca-45fe1b85c275"). InnerVolumeSpecName "kube-api-access-sprn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:47:22.781221 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.781187    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8a8e52b-93a1-42df-acca-45fe1b85c275-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e8a8e52b-93a1-42df-acca-45fe1b85c275" (UID: "e8a8e52b-93a1-42df-acca-45fe1b85c275"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:47:22.781390 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.781297    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e8a8e52b-93a1-42df-acca-45fe1b85c275" (UID: "e8a8e52b-93a1-42df-acca-45fe1b85c275"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:47:22.781390 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.781337    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8a8e52b-93a1-42df-acca-45fe1b85c275-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e8a8e52b-93a1-42df-acca-45fe1b85c275" (UID: "e8a8e52b-93a1-42df-acca-45fe1b85c275"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:47:22.812893 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.812860    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5"]
Jul 31 23:47:22.816142 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:47:22.816113    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod436d6b79_58a5_4a66_9ce7_07f66df1c251.slice/crio-3c17a4fbf7f35f1c9e08dc7628bfff4a7622a3a4464f73b7d32f85257f9ee2c5 WatchSource:0}: Error finding container 3c17a4fbf7f35f1c9e08dc7628bfff4a7622a3a4464f73b7d32f85257f9ee2c5: Status 404 returned error can't find the container with id 3c17a4fbf7f35f1c9e08dc7628bfff4a7622a3a4464f73b7d32f85257f9ee2c5
Jul 31 23:47:22.833810 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.833779    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw"]
Jul 31 23:47:22.837133 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:47:22.837101    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26b85f59_9239_4d95_a3f2_47de1ec9648e.slice/crio-bae1bc554310b60118f13fc46f833428f77cdf574c83278813981a5a47ecff74 WatchSource:0}: Error finding container bae1bc554310b60118f13fc46f833428f77cdf574c83278813981a5a47ecff74: Status 404 returned error can't find the container with id bae1bc554310b60118f13fc46f833428f77cdf574c83278813981a5a47ecff74
Jul 31 23:47:22.870301 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.870267    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8a8e52b-93a1-42df-acca-45fe1b85c275-trusted-ca\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:47:22.870301 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.870293    2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-bound-sa-token\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:47:22.870301 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.870305    2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8a8e52b-93a1-42df-acca-45fe1b85c275-image-registry-private-configuration\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:47:22.870514 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.870319    2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8a8e52b-93a1-42df-acca-45fe1b85c275-installation-pull-secrets\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:47:22.870514 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.870332    2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-certificates\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:47:22.870514 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.870341    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sprn9\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-kube-api-access-sprn9\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:47:22.870514 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:22.870350    2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8a8e52b-93a1-42df-acca-45fe1b85c275-ca-trust-extracted\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:47:23.172485 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.172447    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret\") pod \"global-pull-secret-syncer-4k4dl\" (UID: \"6c1ae62a-0b92-4205-a4bb-5d266930b743\") " pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:47:23.172676 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.172506    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs\") pod \"network-metrics-daemon-tn7kj\" (UID: \"75c8e3ed-6587-4ef9-b3bb-562bf74fec5b\") " pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:47:23.177768 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.177740    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 31 23:47:23.177768 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.177763    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 31 23:47:23.184942 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.184914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c1ae62a-0b92-4205-a4bb-5d266930b743-original-pull-secret\") pod \"global-pull-secret-syncer-4k4dl\" (UID: \"6c1ae62a-0b92-4205-a4bb-5d266930b743\") " pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:47:23.184942 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.184933    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75c8e3ed-6587-4ef9-b3bb-562bf74fec5b-metrics-certs\") pod \"network-metrics-daemon-tn7kj\" (UID: \"75c8e3ed-6587-4ef9-b3bb-562bf74fec5b\") " pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:47:23.207062 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.207035    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v7qx8\""
Jul 31 23:47:23.210746 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.210726    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k4dl"
Jul 31 23:47:23.215150 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.215131    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tn7kj"
Jul 31 23:47:23.276634 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.276591    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87ptm\" (UniqueName: \"kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm\") pod \"network-check-target-frb4r\" (UID: \"1cc3f661-c981-4802-964e-632fab8107c6\") " pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:47:23.279522 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.279491    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 31 23:47:23.291089 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.290896    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 31 23:47:23.301605 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.301546    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87ptm\" (UniqueName: \"kubernetes.io/projected/1cc3f661-c981-4802-964e-632fab8107c6-kube-api-access-87ptm\") pod \"network-check-target-frb4r\" (UID: \"1cc3f661-c981-4802-964e-632fab8107c6\") " pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:47:23.377250 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.376822    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tn7kj"]
Jul 31 23:47:23.380884 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:47:23.380842    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75c8e3ed_6587_4ef9_b3bb_562bf74fec5b.slice/crio-cea1acd560de6971e756d728a1cd96a682991ef4fd9cbfdf652ea222bedbc023 WatchSource:0}: Error finding container cea1acd560de6971e756d728a1cd96a682991ef4fd9cbfdf652ea222bedbc023: Status 404 returned error can't find the container with id cea1acd560de6971e756d728a1cd96a682991ef4fd9cbfdf652ea222bedbc023
Jul 31 23:47:23.392416 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.392348    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4k4dl"]
Jul 31 23:47:23.396550 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:47:23.396516    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c1ae62a_0b92_4205_a4bb_5d266930b743.slice/crio-db125ed81896ab3f821adc5902a10f85780804b9395cc9de06b712a1c23c56f9 WatchSource:0}: Error finding container db125ed81896ab3f821adc5902a10f85780804b9395cc9de06b712a1c23c56f9: Status 404 returned error can't find the container with id db125ed81896ab3f821adc5902a10f85780804b9395cc9de06b712a1c23c56f9
Jul 31 23:47:23.499563 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.499484    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5t5tm\""
Jul 31 23:47:23.508068 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.507732    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:47:23.684311 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.684186    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-frb4r"]
Jul 31 23:47:23.689701 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:47:23.689661    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc3f661_c981_4802_964e_632fab8107c6.slice/crio-b047028c68a0fd88709be894e66d2e383b8f51b657b25e8f27d4e369bbfaef18 WatchSource:0}: Error finding container b047028c68a0fd88709be894e66d2e383b8f51b657b25e8f27d4e369bbfaef18: Status 404 returned error can't find the container with id b047028c68a0fd88709be894e66d2e383b8f51b657b25e8f27d4e369bbfaef18
Jul 31 23:47:23.697291 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.697206    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4k4dl" event={"ID":"6c1ae62a-0b92-4205-a4bb-5d266930b743","Type":"ContainerStarted","Data":"db125ed81896ab3f821adc5902a10f85780804b9395cc9de06b712a1c23c56f9"}
Jul 31 23:47:23.698995 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.698946    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5" event={"ID":"436d6b79-58a5-4a66-9ce7-07f66df1c251","Type":"ContainerStarted","Data":"3c17a4fbf7f35f1c9e08dc7628bfff4a7622a3a4464f73b7d32f85257f9ee2c5"}
Jul 31 23:47:23.700701 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.700616    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tn7kj" event={"ID":"75c8e3ed-6587-4ef9-b3bb-562bf74fec5b","Type":"ContainerStarted","Data":"cea1acd560de6971e756d728a1cd96a682991ef4fd9cbfdf652ea222bedbc023"}
Jul 31 23:47:23.702916 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.702268    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-574d844ff-fdscc"
Jul 31 23:47:23.702916 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.702717    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw" event={"ID":"26b85f59-9239-4d95-a3f2-47de1ec9648e","Type":"ContainerStarted","Data":"bae1bc554310b60118f13fc46f833428f77cdf574c83278813981a5a47ecff74"}
Jul 31 23:47:23.738697 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.738635    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-574d844ff-fdscc"]
Jul 31 23:47:23.740802 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.740747    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-574d844ff-fdscc"]
Jul 31 23:47:23.783161 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:23.783052    2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8a8e52b-93a1-42df-acca-45fe1b85c275-registry-tls\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:47:24.711210 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:24.711090    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-frb4r" event={"ID":"1cc3f661-c981-4802-964e-632fab8107c6","Type":"ContainerStarted","Data":"b047028c68a0fd88709be894e66d2e383b8f51b657b25e8f27d4e369bbfaef18"}
Jul 31 23:47:25.096036 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:25.095609    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:47:25.106039 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:25.105939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c461a2c-8de2-4813-9cfd-45366645a56e-metrics-tls\") pod \"dns-default-x8242\" (UID: \"4c461a2c-8de2-4813-9cfd-45366645a56e\") " pod="openshift-dns/dns-default-x8242"
Jul 31 23:47:25.301774 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:25.301546    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rs996\""
Jul 31 23:47:25.310099 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:25.309798    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x8242"
Jul 31 23:47:25.388973 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:25.388746    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8a8e52b-93a1-42df-acca-45fe1b85c275" path="/var/lib/kubelet/pods/e8a8e52b-93a1-42df-acca-45fe1b85c275/volumes"
Jul 31 23:47:31.554972 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:31.554919    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:47:31.557679 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:31.557648    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dcbx4\" (UID: \"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f\") " pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:47:31.727508 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:31.727475    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fst58\""
Jul 31 23:47:31.735459 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:31.735423    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dcbx4"
Jul 31 23:47:32.290249 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.290210    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x8242"]
Jul 31 23:47:32.326891 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.325776    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dcbx4"]
Jul 31 23:47:32.331358 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:47:32.331324    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c3b0afb_4808_4f97_9ac3_c0a7a13d4b2f.slice/crio-2204ce7f2ca85f6e0b666ace5a96a8d08b97437e6fe5ee1b0a04037d4be49d26 WatchSource:0}: Error finding container 2204ce7f2ca85f6e0b666ace5a96a8d08b97437e6fe5ee1b0a04037d4be49d26: Status 404 returned error can't find the container with id 2204ce7f2ca85f6e0b666ace5a96a8d08b97437e6fe5ee1b0a04037d4be49d26
Jul 31 23:47:32.739167 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.738459    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tn7kj" event={"ID":"75c8e3ed-6587-4ef9-b3bb-562bf74fec5b","Type":"ContainerStarted","Data":"3fd292f203cf3b219cb1b33cf37f3472eabf279ae860af6a168a982e0b40c109"}
Jul 31 23:47:32.739167 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.738603    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tn7kj" event={"ID":"75c8e3ed-6587-4ef9-b3bb-562bf74fec5b","Type":"ContainerStarted","Data":"6e7695fc22a40fb99535fb563e150e7ffc2b23690d26adab74f883f454ff0324"}
Jul 31 23:47:32.741478 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.741440    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw" event={"ID":"26b85f59-9239-4d95-a3f2-47de1ec9648e","Type":"ContainerStarted","Data":"e56f27b1ccef90632ea17d73424afc572701e27a0d3d8931c9ae9943f0fdd74e"}
Jul 31 23:47:32.741624 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.741525    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw" event={"ID":"26b85f59-9239-4d95-a3f2-47de1ec9648e","Type":"ContainerStarted","Data":"b57e06916b0e5bd18ba0c7049778ac443d4b8c65bffee0aa7ed2942165a64169"}
Jul 31 23:47:32.741624 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.741546    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw" event={"ID":"26b85f59-9239-4d95-a3f2-47de1ec9648e","Type":"ContainerStarted","Data":"40ab7f41384e9d8a3d6245ab223053e92e9551218e31bee7495b83ade7c97559"}
Jul 31 23:47:32.744287 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.744252    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dcbx4" event={"ID":"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f","Type":"ContainerStarted","Data":"2e3ecb4769c6da20d2176964324a82625d8cbf9d18638ce7a9d0e0c6949e7f9c"}
Jul 31 23:47:32.744455 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.744321    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dcbx4" event={"ID":"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f","Type":"ContainerStarted","Data":"2204ce7f2ca85f6e0b666ace5a96a8d08b97437e6fe5ee1b0a04037d4be49d26"}
Jul 31 23:47:32.746439 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.746406    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x8242" event={"ID":"4c461a2c-8de2-4813-9cfd-45366645a56e","Type":"ContainerStarted","Data":"a592202624dc737868b21285955c3a10759c68a038bbc1cab14a803f46f545ed"}
Jul 31 23:47:32.748381 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.748349    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4k4dl" event={"ID":"6c1ae62a-0b92-4205-a4bb-5d266930b743","Type":"ContainerStarted","Data":"b5f28b3428b63cc8e8ee699d0dcff9d337695301c635cae66c1410c1f344f0a4"}
Jul 31 23:47:32.750458 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.750165    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5" event={"ID":"436d6b79-58a5-4a66-9ce7-07f66df1c251","Type":"ContainerStarted","Data":"f9a883dcc874f5c09a65802225e7905833ae9610715323dab00e8b9366632c90"}
Jul 31 23:47:32.750723 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.750679    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5"
Jul 31 23:47:32.754005 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.753977    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5"
Jul 31 23:47:32.754893 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.754839    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tn7kj" podStartSLOduration=66.037794498 podStartE2EDuration="1m14.754821243s" totalImagesPullingTime="8.717026745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:32.754215665 +0000 UTC m=+75.933672401" watchObservedRunningTime="2026-07-31 23:47:32.754821243 +0000 UTC m=+75.934277988"
Jul 31 23:47:32.755834 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.755799    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-frb4r" event={"ID":"1cc3f661-c981-4802-964e-632fab8107c6","Type":"ContainerStarted","Data":"07c03f2a44502dd684c7f429485fe200c2cec06873494e6665383bfda9af9ccf"}
Jul 31 23:47:32.756000 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.755949    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:47:32.770640 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.770564    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84897dbdb8-dh7l5" podStartSLOduration=1.465072954 podStartE2EDuration="10.770540164s" totalImagesPullingTime="9.30546721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:32.769568181 +0000 UTC m=+75.949024929" watchObservedRunningTime="2026-07-31 23:47:32.770540164 +0000 UTC m=+75.949996888"
Jul 31 23:47:32.784765 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.784694    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4k4dl" podStartSLOduration=66.067370736 podStartE2EDuration="1m14.78467239s" totalImagesPullingTime="8.717301654s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:32.784248589 +0000 UTC m=+75.963705334" watchObservedRunningTime="2026-07-31 23:47:32.78467239 +0000 UTC m=+75.964129135"
Jul 31 23:47:32.803578 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.803518    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-866b48d95-xbsqw" podStartSLOduration=1.523105668 podStartE2EDuration="10.803501681s" totalImagesPullingTime="9.280396013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:32.803124255 +0000 UTC m=+75.982581000" watchObservedRunningTime="2026-07-31 23:47:32.803501681 +0000 UTC m=+75.982958425"
Jul 31 23:47:32.818086 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:32.818037    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-frb4r" podStartSLOduration=66.396754077 podStartE2EDuration="1m14.818013231s" totalImagesPullingTime="8.421259154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:32.817868927 +0000 UTC m=+75.997325671" watchObservedRunningTime="2026-07-31 23:47:32.818013231 +0000 UTC m=+75.997470312"
Jul 31 23:47:33.703503 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.703469    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f876b6c9f-t2z27"]
Jul 31 23:47:33.706711 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.706691    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.709230 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.709207    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 31 23:47:33.709527 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.709507    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 31 23:47:33.709638 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.709545    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 31 23:47:33.710469 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.710447    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 31 23:47:33.710579 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.710526    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 31 23:47:33.710579 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.710539    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 31 23:47:33.710579 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.710530    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 31 23:47:33.710794 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.710779    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z24jv\""
Jul 31 23:47:33.715760 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.715741    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 31 23:47:33.718081 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.718060    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f876b6c9f-t2z27"]
Jul 31 23:47:33.762005 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.760659    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dcbx4" event={"ID":"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f","Type":"ContainerStarted","Data":"12adf874c102377d12ec80b5ea766613bd0c5dbb4e1dd7f88afbe9f1f30e8042"}
Jul 31 23:47:33.777554 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.777515    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-oauth-config\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.777686 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.777580    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-service-ca\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.777686 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.777615    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcs92\" (UniqueName: \"kubernetes.io/projected/d0447f18-e6b7-4cc0-a571-b4c53a60221b-kube-api-access-bcs92\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.777686 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.777639    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-config\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.777849 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.777727    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-trusted-ca-bundle\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.777849 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.777785    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-oauth-serving-cert\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.777924 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.777871    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-serving-cert\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.879415 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.878533    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-oauth-config\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.879415 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.878627    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-service-ca\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.879415 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.878672    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcs92\" (UniqueName: \"kubernetes.io/projected/d0447f18-e6b7-4cc0-a571-b4c53a60221b-kube-api-access-bcs92\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.879415 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.878700    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-config\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.879415 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.878823    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-trusted-ca-bundle\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.879415 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.878857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-oauth-serving-cert\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.879832 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.879444    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-serving-cert\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.879903 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.879840    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-config\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.882015 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.880597    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-trusted-ca-bundle\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.882015 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.881160    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-service-ca\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.882015 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.881706    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-oauth-serving-cert\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.884632 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.884586    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-serving-cert\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.888160 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.888133    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcs92\" (UniqueName: \"kubernetes.io/projected/d0447f18-e6b7-4cc0-a571-b4c53a60221b-kube-api-access-bcs92\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:33.889152 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:33.889102    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-oauth-config\") pod \"console-5f876b6c9f-t2z27\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") " pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:34.015917 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:34.015819    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:34.161574 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:34.161538    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f876b6c9f-t2z27"]
Jul 31 23:47:34.165604 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:47:34.165569    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0447f18_e6b7_4cc0_a571_b4c53a60221b.slice/crio-0952d50dbf7bc31531278e869046f938b29cf014b0cc2ffd36e1e2366a828510 WatchSource:0}: Error finding container 0952d50dbf7bc31531278e869046f938b29cf014b0cc2ffd36e1e2366a828510: Status 404 returned error can't find the container with id 0952d50dbf7bc31531278e869046f938b29cf014b0cc2ffd36e1e2366a828510
Jul 31 23:47:34.767011 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:34.766974    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dcbx4" event={"ID":"4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f","Type":"ContainerStarted","Data":"a4fc807c2f8fca1d30dab464ff16dbeea806bb38bc7d436730ad4622a0fb66cc"}
Jul 31 23:47:34.769163 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:34.769136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x8242" event={"ID":"4c461a2c-8de2-4813-9cfd-45366645a56e","Type":"ContainerStarted","Data":"1351760cb86e21731e8a5dcd2ea09460984de1e85735f323a654fdb69ad9d4d5"}
Jul 31 23:47:34.769307 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:34.769193    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-x8242"
Jul 31 23:47:34.769307 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:34.769216    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x8242" event={"ID":"4c461a2c-8de2-4813-9cfd-45366645a56e","Type":"ContainerStarted","Data":"0913690e75a8b50eaea1f8c6f801a7a753d8b4668644ff718679ea80b85fc030"}
Jul 31 23:47:34.770525 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:34.770505    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f876b6c9f-t2z27" event={"ID":"d0447f18-e6b7-4cc0-a571-b4c53a60221b","Type":"ContainerStarted","Data":"0952d50dbf7bc31531278e869046f938b29cf014b0cc2ffd36e1e2366a828510"}
Jul 31 23:47:34.803814 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:34.803744    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-x8242" podStartSLOduration=40.342201776 podStartE2EDuration="41.803722585s" totalImagesPullingTime="1.461520809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:46:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:34.803056677 +0000 UTC m=+77.982513422" watchObservedRunningTime="2026-07-31 23:47:34.803722585 +0000 UTC m=+77.983179330"
Jul 31 23:47:34.804624 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:34.804578    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dcbx4" podStartSLOduration=33.59463383 podStartE2EDuration="35.804562251s" totalImagesPullingTime="2.209928421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:34.787441426 +0000 UTC m=+77.966898171" watchObservedRunningTime="2026-07-31 23:47:34.804562251 +0000 UTC m=+77.984018994"
Jul 31 23:47:37.782589 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:37.782549    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f876b6c9f-t2z27" event={"ID":"d0447f18-e6b7-4cc0-a571-b4c53a60221b","Type":"ContainerStarted","Data":"93dbab87219d25555936cb9c25b456a3f00a77fa1073478a93bdad28c9852fd7"}
Jul 31 23:47:37.808915 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:37.808857    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f876b6c9f-t2z27" podStartSLOduration=2.121148376 podStartE2EDuration="4.80884017s" totalImagesPullingTime="2.687691794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:37.807939893 +0000 UTC m=+80.987396638" watchObservedRunningTime="2026-07-31 23:47:37.80884017 +0000 UTC m=+80.988296913"
Jul 31 23:47:37.907947 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:37.907912    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-727b8"]
Jul 31 23:47:37.910876 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:37.910858    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:37.913708 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:37.913686    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 31 23:47:37.914591 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:37.914572    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 31 23:47:37.914591 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:37.914587    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 31 23:47:37.914743 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:37.914616    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 31 23:47:37.914743 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:37.914591    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 31 23:47:37.914743 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:37.914693    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 31 23:47:37.914928 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:37.914912    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cmkpk\""
Jul 31 23:47:38.012577 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.012540    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-tls\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.012758 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.012590    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e5618be-8358-4dba-aa6e-3078a1f8c05b-sys\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.012758 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.012622    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2lst\" (UniqueName: \"kubernetes.io/projected/7e5618be-8358-4dba-aa6e-3078a1f8c05b-kube-api-access-w2lst\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.012758 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.012667    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.012758 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.012728    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-textfile\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.012758 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.012757    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e5618be-8358-4dba-aa6e-3078a1f8c05b-metrics-client-ca\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.012940 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.012781    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-accelerators-collector-config\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.012940 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.012800    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-wtmp\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.012940 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.012821    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e5618be-8358-4dba-aa6e-3078a1f8c05b-root\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.113234 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.113142    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e5618be-8358-4dba-aa6e-3078a1f8c05b-sys\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.113234 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.113181    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2lst\" (UniqueName: \"kubernetes.io/projected/7e5618be-8358-4dba-aa6e-3078a1f8c05b-kube-api-access-w2lst\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.113234 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.113201    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.113491 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.113269    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e5618be-8358-4dba-aa6e-3078a1f8c05b-sys\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.113491 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.113316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-textfile\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.113491 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.113341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e5618be-8358-4dba-aa6e-3078a1f8c05b-metrics-client-ca\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.113491 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.113361    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-accelerators-collector-config\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.113491 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.113383    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-wtmp\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.113491 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.113407    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e5618be-8358-4dba-aa6e-3078a1f8c05b-root\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.113491 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.113424    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-tls\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.113800 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:38.113532    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 31 23:47:38.113800 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:47:38.113602    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-tls podName:7e5618be-8358-4dba-aa6e-3078a1f8c05b nodeName:}" failed. No retries permitted until 2026-07-31 23:47:38.613581904 +0000 UTC m=+81.793038626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-tls") pod "node-exporter-727b8" (UID: "7e5618be-8358-4dba-aa6e-3078a1f8c05b") : secret "node-exporter-tls" not found
Jul 31 23:47:38.113800 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.113679    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-wtmp\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.114031 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.113803    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-textfile\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.114031 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.113836    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e5618be-8358-4dba-aa6e-3078a1f8c05b-root\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.114031 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.113980    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-accelerators-collector-config\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.114176 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.114067    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e5618be-8358-4dba-aa6e-3078a1f8c05b-metrics-client-ca\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.115597 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.115578    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.123611 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.123588    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2lst\" (UniqueName: \"kubernetes.io/projected/7e5618be-8358-4dba-aa6e-3078a1f8c05b-kube-api-access-w2lst\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.617587 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.617547    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-tls\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.619887 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.619860    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e5618be-8358-4dba-aa6e-3078a1f8c05b-node-exporter-tls\") pod \"node-exporter-727b8\" (UID: \"7e5618be-8358-4dba-aa6e-3078a1f8c05b\") " pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.819557 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:38.819519    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-727b8"
Jul 31 23:47:38.831326 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:47:38.831293    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e5618be_8358_4dba_aa6e_3078a1f8c05b.slice/crio-f57df086fc6990389941ed430f631e96f49e4f41750b7aa8c41749ac6e5b146f WatchSource:0}: Error finding container f57df086fc6990389941ed430f631e96f49e4f41750b7aa8c41749ac6e5b146f: Status 404 returned error can't find the container with id f57df086fc6990389941ed430f631e96f49e4f41750b7aa8c41749ac6e5b146f
Jul 31 23:47:39.790846 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:39.790808    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-727b8" event={"ID":"7e5618be-8358-4dba-aa6e-3078a1f8c05b","Type":"ContainerStarted","Data":"6a0835417e186ce0c7470c2621e821def556e4b9617671a14c4ed53920b29b69"}
Jul 31 23:47:39.790977 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:39.790889    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-727b8" event={"ID":"7e5618be-8358-4dba-aa6e-3078a1f8c05b","Type":"ContainerStarted","Data":"f57df086fc6990389941ed430f631e96f49e4f41750b7aa8c41749ac6e5b146f"}
Jul 31 23:47:40.796028 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:40.795993    2581 generic.go:356] "Generic (PLEG): container finished" podID="7e5618be-8358-4dba-aa6e-3078a1f8c05b" containerID="6a0835417e186ce0c7470c2621e821def556e4b9617671a14c4ed53920b29b69" exitCode=0
Jul 31 23:47:40.796414 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:40.796058    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-727b8" event={"ID":"7e5618be-8358-4dba-aa6e-3078a1f8c05b","Type":"ContainerDied","Data":"6a0835417e186ce0c7470c2621e821def556e4b9617671a14c4ed53920b29b69"}
Jul 31 23:47:41.801298 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:41.801262    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-727b8" event={"ID":"7e5618be-8358-4dba-aa6e-3078a1f8c05b","Type":"ContainerStarted","Data":"f3cab9c58cf23920c2a80b827c3b090f9ff341a67a012501d773d194e9c94752"}
Jul 31 23:47:41.801794 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:41.801329    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-727b8" event={"ID":"7e5618be-8358-4dba-aa6e-3078a1f8c05b","Type":"ContainerStarted","Data":"3667bc5f996921d7357c8a967b9dbf70969410c4dd60fafdd2533b8d8401f36e"}
Jul 31 23:47:41.822174 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:41.822121    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-727b8" podStartSLOduration=3.932878754 podStartE2EDuration="4.822104062s" totalImagesPullingTime="821.310595ms" totalInitContainerRuntime="67.914713ms" isStatefulPod=true podCreationTimestamp="2026-07-31 23:47:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:41.820387057 +0000 UTC m=+84.999843802" watchObservedRunningTime="2026-07-31 23:47:41.822104062 +0000 UTC m=+85.001560818"
Jul 31 23:47:44.016713 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:44.016672    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:44.017134 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:44.016763    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:44.021703 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:44.021678    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:44.776880 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:44.776849    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x8242"
Jul 31 23:47:44.815510 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:44.815477    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:47:53.923914 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:47:53.923876    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f876b6c9f-t2z27"]
Jul 31 23:48:03.763601 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:03.763570    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-frb4r"
Jul 31 23:48:18.943383 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:18.943321    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f876b6c9f-t2z27" podUID="d0447f18-e6b7-4cc0-a571-b4c53a60221b" containerName="console" containerID="cri-o://93dbab87219d25555936cb9c25b456a3f00a77fa1073478a93bdad28c9852fd7" gracePeriod=15
Jul 31 23:48:19.175889 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.175864    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:48:19.205275 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.205195    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-serving-cert\") pod \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") "
Jul 31 23:48:19.205275 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.205235    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-oauth-config\") pod \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") "
Jul 31 23:48:19.205275 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.205263    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcs92\" (UniqueName: \"kubernetes.io/projected/d0447f18-e6b7-4cc0-a571-b4c53a60221b-kube-api-access-bcs92\") pod \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") "
Jul 31 23:48:19.205527 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.205313    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-oauth-serving-cert\") pod \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") "
Jul 31 23:48:19.205527 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.205330    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-trusted-ca-bundle\") pod \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") "
Jul 31 23:48:19.205527 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.205357    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-config\") pod \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") "
Jul 31 23:48:19.205527 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.205382    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-service-ca\") pod \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\" (UID: \"d0447f18-e6b7-4cc0-a571-b4c53a60221b\") "
Jul 31 23:48:19.205841 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.205796    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-service-ca" (OuterVolumeSpecName: "service-ca") pod "d0447f18-e6b7-4cc0-a571-b4c53a60221b" (UID: "d0447f18-e6b7-4cc0-a571-b4c53a60221b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:48:19.205841 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.205819    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d0447f18-e6b7-4cc0-a571-b4c53a60221b" (UID: "d0447f18-e6b7-4cc0-a571-b4c53a60221b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:48:19.206051 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.205887    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-config" (OuterVolumeSpecName: "console-config") pod "d0447f18-e6b7-4cc0-a571-b4c53a60221b" (UID: "d0447f18-e6b7-4cc0-a571-b4c53a60221b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:48:19.206117 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.206074    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d0447f18-e6b7-4cc0-a571-b4c53a60221b" (UID: "d0447f18-e6b7-4cc0-a571-b4c53a60221b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:48:19.207486 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.207458    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d0447f18-e6b7-4cc0-a571-b4c53a60221b" (UID: "d0447f18-e6b7-4cc0-a571-b4c53a60221b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:48:19.207599 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.207497    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d0447f18-e6b7-4cc0-a571-b4c53a60221b" (UID: "d0447f18-e6b7-4cc0-a571-b4c53a60221b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:48:19.207777 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.207749    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0447f18-e6b7-4cc0-a571-b4c53a60221b-kube-api-access-bcs92" (OuterVolumeSpecName: "kube-api-access-bcs92") pod "d0447f18-e6b7-4cc0-a571-b4c53a60221b" (UID: "d0447f18-e6b7-4cc0-a571-b4c53a60221b"). InnerVolumeSpecName "kube-api-access-bcs92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:48:19.305871 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.305838    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcs92\" (UniqueName: \"kubernetes.io/projected/d0447f18-e6b7-4cc0-a571-b4c53a60221b-kube-api-access-bcs92\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:48:19.305871 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.305866    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-oauth-serving-cert\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:48:19.305871 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.305876    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-trusted-ca-bundle\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:48:19.306177 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.305885    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-config\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:48:19.306177 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.305895    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0447f18-e6b7-4cc0-a571-b4c53a60221b-service-ca\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:48:19.306177 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.305904    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-serving-cert\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:48:19.306177 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.305912    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0447f18-e6b7-4cc0-a571-b4c53a60221b-console-oauth-config\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:48:19.933372 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.933337    2581 generic.go:356] "Generic (PLEG): container finished" podID="d0447f18-e6b7-4cc0-a571-b4c53a60221b" containerID="93dbab87219d25555936cb9c25b456a3f00a77fa1073478a93bdad28c9852fd7" exitCode=0
Jul 31 23:48:19.933541 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.933383    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f876b6c9f-t2z27"
Jul 31 23:48:19.933541 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.933406    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f876b6c9f-t2z27" event={"ID":"d0447f18-e6b7-4cc0-a571-b4c53a60221b","Type":"ContainerDied","Data":"93dbab87219d25555936cb9c25b456a3f00a77fa1073478a93bdad28c9852fd7"}
Jul 31 23:48:19.933541 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.933446    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f876b6c9f-t2z27" event={"ID":"d0447f18-e6b7-4cc0-a571-b4c53a60221b","Type":"ContainerDied","Data":"0952d50dbf7bc31531278e869046f938b29cf014b0cc2ffd36e1e2366a828510"}
Jul 31 23:48:19.933541 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.933465    2581 scope.go:117] "RemoveContainer" containerID="93dbab87219d25555936cb9c25b456a3f00a77fa1073478a93bdad28c9852fd7"
Jul 31 23:48:19.941119 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.941100    2581 scope.go:117] "RemoveContainer" containerID="93dbab87219d25555936cb9c25b456a3f00a77fa1073478a93bdad28c9852fd7"
Jul 31 23:48:19.941380 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:48:19.941363    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93dbab87219d25555936cb9c25b456a3f00a77fa1073478a93bdad28c9852fd7\": container with ID starting with 93dbab87219d25555936cb9c25b456a3f00a77fa1073478a93bdad28c9852fd7 not found: ID does not exist" containerID="93dbab87219d25555936cb9c25b456a3f00a77fa1073478a93bdad28c9852fd7"
Jul 31 23:48:19.941449 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.941387    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93dbab87219d25555936cb9c25b456a3f00a77fa1073478a93bdad28c9852fd7"} err="failed to get container status \"93dbab87219d25555936cb9c25b456a3f00a77fa1073478a93bdad28c9852fd7\": rpc error: code = NotFound desc = could not find container \"93dbab87219d25555936cb9c25b456a3f00a77fa1073478a93bdad28c9852fd7\": container with ID starting with 93dbab87219d25555936cb9c25b456a3f00a77fa1073478a93bdad28c9852fd7 not found: ID does not exist"
Jul 31 23:48:19.948773 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.948743    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f876b6c9f-t2z27"]
Jul 31 23:48:19.951927 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:19.951906    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f876b6c9f-t2z27"]
Jul 31 23:48:21.386807 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:48:21.386766    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0447f18-e6b7-4cc0-a571-b4c53a60221b" path="/var/lib/kubelet/pods/d0447f18-e6b7-4cc0-a571-b4c53a60221b/volumes"
Jul 31 23:49:04.587495 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.587463    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b5bd556f4-b6974"]
Jul 31 23:49:04.587935 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.587722    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0447f18-e6b7-4cc0-a571-b4c53a60221b" containerName="console"
Jul 31 23:49:04.587935 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.587736    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0447f18-e6b7-4cc0-a571-b4c53a60221b" containerName="console"
Jul 31 23:49:04.587935 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.587776    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0447f18-e6b7-4cc0-a571-b4c53a60221b" containerName="console"
Jul 31 23:49:04.592126 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.592108    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.594950 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.594929    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 31 23:49:04.596133 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.596111    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 31 23:49:04.596133 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.596124    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 31 23:49:04.596352 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.596124    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 31 23:49:04.596352 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.596178    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z24jv\""
Jul 31 23:49:04.596352 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.596239    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 31 23:49:04.596476 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.596461    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 31 23:49:04.596516 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.596494    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 31 23:49:04.600201 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.600178    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 31 23:49:04.602084 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.602032    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b5bd556f4-b6974"]
Jul 31 23:49:04.729680 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.729622    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-oauth-config\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.729680 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.729677    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-service-ca\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.729908 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.729752    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-serving-cert\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.729908 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.729792    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-trusted-ca-bundle\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.729908 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.729826    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-oauth-serving-cert\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.729908 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.729878    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-config\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.729908 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.729900    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59gwr\" (UniqueName: \"kubernetes.io/projected/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-kube-api-access-59gwr\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.830704 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.830671    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-oauth-config\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.830704 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.830708    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-service-ca\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.830909 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.830735    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-serving-cert\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.830909 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.830749    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-trusted-ca-bundle\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.830909 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.830858    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-oauth-serving-cert\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.830909 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.830902    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-config\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.831170 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.831074    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59gwr\" (UniqueName: \"kubernetes.io/projected/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-kube-api-access-59gwr\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.831625 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.831599    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-service-ca\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.831715 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.831624    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-oauth-serving-cert\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.831715 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.831648    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-config\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.831715 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.831656    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-trusted-ca-bundle\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.833291 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.833258    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-oauth-config\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.833453 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.833435    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-serving-cert\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.840349 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.840286    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59gwr\" (UniqueName: \"kubernetes.io/projected/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-kube-api-access-59gwr\") pod \"console-6b5bd556f4-b6974\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") " pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:04.901152 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:04.901115    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:05.028672 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:05.028639    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b5bd556f4-b6974"]
Jul 31 23:49:05.031598 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:49:05.031568    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0512ca4_d684_4f5b_b8bf_25e9b5e09cc0.slice/crio-fdcae4fcec73886731f69620b19ea580429ddaf9aeba1e62d158130c934ad245 WatchSource:0}: Error finding container fdcae4fcec73886731f69620b19ea580429ddaf9aeba1e62d158130c934ad245: Status 404 returned error can't find the container with id fdcae4fcec73886731f69620b19ea580429ddaf9aeba1e62d158130c934ad245
Jul 31 23:49:05.080233 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:05.080200    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b5bd556f4-b6974" event={"ID":"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0","Type":"ContainerStarted","Data":"fdcae4fcec73886731f69620b19ea580429ddaf9aeba1e62d158130c934ad245"}
Jul 31 23:49:06.084737 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:06.084697    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b5bd556f4-b6974" event={"ID":"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0","Type":"ContainerStarted","Data":"622cabac131b8358ca2d8e222afe9c4aee8f91c52599dfb3c059993aa85b43c9"}
Jul 31 23:49:06.103555 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:06.103498    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b5bd556f4-b6974" podStartSLOduration=2.103480658 podStartE2EDuration="2.103480658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:49:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:49:06.102427024 +0000 UTC m=+169.281883780" watchObservedRunningTime="2026-07-31 23:49:06.103480658 +0000 UTC m=+169.282937402"
Jul 31 23:49:14.901509 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:14.901404    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:14.901509 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:14.901500    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:14.906111 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:14.906091    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:15.119372 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.119343    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:15.231509 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.231429    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67d7b7779-n5rt6"]
Jul 31 23:49:15.234407 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.234385    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.247375 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.247349    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67d7b7779-n5rt6"]
Jul 31 23:49:15.404417 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.404379    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-config\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.404417 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.404416    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-trusted-ca-bundle\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.404635 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.404442    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdhdd\" (UniqueName: \"kubernetes.io/projected/e44df733-fe2b-459b-a84e-a27acca6e2cf-kube-api-access-zdhdd\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.404635 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.404504    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-oauth-config\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.404635 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.404542    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-serving-cert\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.404635 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.404560    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-service-ca\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.404635 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.404583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-oauth-serving-cert\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.505831 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.505732    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-config\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.505831 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.505777    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-trusted-ca-bundle\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.505831 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.505798    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdhdd\" (UniqueName: \"kubernetes.io/projected/e44df733-fe2b-459b-a84e-a27acca6e2cf-kube-api-access-zdhdd\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.505831 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.505831    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-oauth-config\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.506630 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.506468    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-serving-cert\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.507060 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.507033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-service-ca\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.511487 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.507228    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-service-ca\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.511487 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.507155    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-trusted-ca-bundle\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.511487 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.507232    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-oauth-serving-cert\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.511487 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.507065    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-config\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.511487 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.507788    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-oauth-serving-cert\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.511487 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.509169    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-oauth-config\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.511487 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.509298    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-serving-cert\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.514583 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.514554    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdhdd\" (UniqueName: \"kubernetes.io/projected/e44df733-fe2b-459b-a84e-a27acca6e2cf-kube-api-access-zdhdd\") pod \"console-67d7b7779-n5rt6\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") " pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.543472 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.543416    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:15.673622 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:15.673589    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67d7b7779-n5rt6"]
Jul 31 23:49:15.677797 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:49:15.677760    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode44df733_fe2b_459b_a84e_a27acca6e2cf.slice/crio-6bbae17bbea3900ca132b3e4209b53ef11155f4d79215a4748ef0a7d3211939d WatchSource:0}: Error finding container 6bbae17bbea3900ca132b3e4209b53ef11155f4d79215a4748ef0a7d3211939d: Status 404 returned error can't find the container with id 6bbae17bbea3900ca132b3e4209b53ef11155f4d79215a4748ef0a7d3211939d
Jul 31 23:49:16.120629 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:16.120584    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d7b7779-n5rt6" event={"ID":"e44df733-fe2b-459b-a84e-a27acca6e2cf","Type":"ContainerStarted","Data":"2293eaa72ace470910682d1c8baf21ef9a9b6bedd869173c9d996f402e2f1e3c"}
Jul 31 23:49:16.121038 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:16.120650    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d7b7779-n5rt6" event={"ID":"e44df733-fe2b-459b-a84e-a27acca6e2cf","Type":"ContainerStarted","Data":"6bbae17bbea3900ca132b3e4209b53ef11155f4d79215a4748ef0a7d3211939d"}
Jul 31 23:49:16.138372 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:16.138325    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67d7b7779-n5rt6" podStartSLOduration=1.138309871 podStartE2EDuration="1.138309871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:49:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:49:16.137155439 +0000 UTC m=+179.316612195" watchObservedRunningTime="2026-07-31 23:49:16.138309871 +0000 UTC m=+179.317766615"
Jul 31 23:49:25.543574 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:25.543533    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:25.543994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:25.543685    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:25.548493 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:25.548469    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:26.158484 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:26.158456    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:49:26.202352 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:26.202314    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b5bd556f4-b6974"]
Jul 31 23:49:51.227066 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.227003    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b5bd556f4-b6974" podUID="a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0" containerName="console" containerID="cri-o://622cabac131b8358ca2d8e222afe9c4aee8f91c52599dfb3c059993aa85b43c9" gracePeriod=15
Jul 31 23:49:51.466750 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.466722    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:51.570391 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.570293    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-oauth-config\") pod \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") "
Jul 31 23:49:51.570391 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.570336    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-serving-cert\") pod \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") "
Jul 31 23:49:51.570391 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.570363    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-trusted-ca-bundle\") pod \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") "
Jul 31 23:49:51.570680 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.570394    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-config\") pod \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") "
Jul 31 23:49:51.570680 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.570422    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-oauth-serving-cert\") pod \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") "
Jul 31 23:49:51.570680 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.570453    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-service-ca\") pod \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") "
Jul 31 23:49:51.570680 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.570519    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59gwr\" (UniqueName: \"kubernetes.io/projected/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-kube-api-access-59gwr\") pod \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\" (UID: \"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0\") "
Jul 31 23:49:51.571151 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.570937    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-config" (OuterVolumeSpecName: "console-config") pod "a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0" (UID: "a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:49:51.571151 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.570948    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0" (UID: "a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:49:51.571151 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.571013    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0" (UID: "a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:49:51.571151 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.571109    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-service-ca" (OuterVolumeSpecName: "service-ca") pod "a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0" (UID: "a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:49:51.572651 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.572627    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-kube-api-access-59gwr" (OuterVolumeSpecName: "kube-api-access-59gwr") pod "a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0" (UID: "a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0"). InnerVolumeSpecName "kube-api-access-59gwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:49:51.572756 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.572660    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0" (UID: "a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:49:51.572756 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.572720    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0" (UID: "a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:49:51.671372 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.671328    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-oauth-config\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:49:51.671372 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.671362    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-serving-cert\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:49:51.671372 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.671375    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-trusted-ca-bundle\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:49:51.671604 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.671386    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-console-config\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:49:51.671604 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.671399    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-oauth-serving-cert\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:49:51.671604 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.671410    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-service-ca\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:49:51.671604 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:51.671422    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59gwr\" (UniqueName: \"kubernetes.io/projected/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0-kube-api-access-59gwr\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:49:52.244436 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:52.244395    2581 generic.go:356] "Generic (PLEG): container finished" podID="a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0" containerID="622cabac131b8358ca2d8e222afe9c4aee8f91c52599dfb3c059993aa85b43c9" exitCode=0
Jul 31 23:49:52.244888 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:52.244446    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b5bd556f4-b6974"
Jul 31 23:49:52.244888 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:52.244462    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b5bd556f4-b6974" event={"ID":"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0","Type":"ContainerDied","Data":"622cabac131b8358ca2d8e222afe9c4aee8f91c52599dfb3c059993aa85b43c9"}
Jul 31 23:49:52.244888 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:52.244511    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b5bd556f4-b6974" event={"ID":"a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0","Type":"ContainerDied","Data":"fdcae4fcec73886731f69620b19ea580429ddaf9aeba1e62d158130c934ad245"}
Jul 31 23:49:52.244888 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:52.244540    2581 scope.go:117] "RemoveContainer" containerID="622cabac131b8358ca2d8e222afe9c4aee8f91c52599dfb3c059993aa85b43c9"
Jul 31 23:49:52.252733 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:52.252704    2581 scope.go:117] "RemoveContainer" containerID="622cabac131b8358ca2d8e222afe9c4aee8f91c52599dfb3c059993aa85b43c9"
Jul 31 23:49:52.253033 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:49:52.253008    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"622cabac131b8358ca2d8e222afe9c4aee8f91c52599dfb3c059993aa85b43c9\": container with ID starting with 622cabac131b8358ca2d8e222afe9c4aee8f91c52599dfb3c059993aa85b43c9 not found: ID does not exist" containerID="622cabac131b8358ca2d8e222afe9c4aee8f91c52599dfb3c059993aa85b43c9"
Jul 31 23:49:52.253110 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:52.253041    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"622cabac131b8358ca2d8e222afe9c4aee8f91c52599dfb3c059993aa85b43c9"} err="failed to get container status \"622cabac131b8358ca2d8e222afe9c4aee8f91c52599dfb3c059993aa85b43c9\": rpc error: code = NotFound desc = could not find container \"622cabac131b8358ca2d8e222afe9c4aee8f91c52599dfb3c059993aa85b43c9\": container with ID starting with 622cabac131b8358ca2d8e222afe9c4aee8f91c52599dfb3c059993aa85b43c9 not found: ID does not exist"
Jul 31 23:49:52.264448 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:52.264426    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b5bd556f4-b6974"]
Jul 31 23:49:52.267387 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:52.267364    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b5bd556f4-b6974"]
Jul 31 23:49:53.386936 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:49:53.386898    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0" path="/var/lib/kubelet/pods/a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0/volumes"
Jul 31 23:50:30.547397 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.547362    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq"]
Jul 31 23:50:30.547883 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.547585    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0" containerName="console"
Jul 31 23:50:30.547883 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.547596    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0" containerName="console"
Jul 31 23:50:30.547883 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.547667    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0512ca4-d684-4f5b-b8bf-25e9b5e09cc0" containerName="console"
Jul 31 23:50:30.551705 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.551686    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq"
Jul 31 23:50:30.554929 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.554902    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 31 23:50:30.554929 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.554919    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 31 23:50:30.555886 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.555872    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-cshng\""
Jul 31 23:50:30.560824 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.560797    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq"]
Jul 31 23:50:30.650748 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.650707    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p7qv\" (UniqueName: \"kubernetes.io/projected/091768bf-2dae-4e4f-aa46-52b2ae8c140d-kube-api-access-5p7qv\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq\" (UID: \"091768bf-2dae-4e4f-aa46-52b2ae8c140d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq"
Jul 31 23:50:30.650988 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.650772    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/091768bf-2dae-4e4f-aa46-52b2ae8c140d-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq\" (UID: \"091768bf-2dae-4e4f-aa46-52b2ae8c140d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq"
Jul 31 23:50:30.650988 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.650859    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/091768bf-2dae-4e4f-aa46-52b2ae8c140d-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq\" (UID: \"091768bf-2dae-4e4f-aa46-52b2ae8c140d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq"
Jul 31 23:50:30.752191 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.752144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/091768bf-2dae-4e4f-aa46-52b2ae8c140d-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq\" (UID: \"091768bf-2dae-4e4f-aa46-52b2ae8c140d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq"
Jul 31 23:50:30.752374 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.752209    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/091768bf-2dae-4e4f-aa46-52b2ae8c140d-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq\" (UID: \"091768bf-2dae-4e4f-aa46-52b2ae8c140d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq"
Jul 31 23:50:30.752374 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.752274    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p7qv\" (UniqueName: \"kubernetes.io/projected/091768bf-2dae-4e4f-aa46-52b2ae8c140d-kube-api-access-5p7qv\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq\" (UID: \"091768bf-2dae-4e4f-aa46-52b2ae8c140d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq"
Jul 31 23:50:30.752538 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.752515    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/091768bf-2dae-4e4f-aa46-52b2ae8c140d-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq\" (UID: \"091768bf-2dae-4e4f-aa46-52b2ae8c140d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq"
Jul 31 23:50:30.752592 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.752570    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/091768bf-2dae-4e4f-aa46-52b2ae8c140d-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq\" (UID: \"091768bf-2dae-4e4f-aa46-52b2ae8c140d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq"
Jul 31 23:50:30.761290 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.761256    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p7qv\" (UniqueName: \"kubernetes.io/projected/091768bf-2dae-4e4f-aa46-52b2ae8c140d-kube-api-access-5p7qv\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq\" (UID: \"091768bf-2dae-4e4f-aa46-52b2ae8c140d\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq"
Jul 31 23:50:30.861430 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.861334    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq"
Jul 31 23:50:30.993424 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:30.990840    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq"]
Jul 31 23:50:30.996305 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:50:30.996275    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091768bf_2dae_4e4f_aa46_52b2ae8c140d.slice/crio-b6720fcb95dabcc3816e1226be9f8d33a3e7e9b03d228299a7ec129633c079d7 WatchSource:0}: Error finding container b6720fcb95dabcc3816e1226be9f8d33a3e7e9b03d228299a7ec129633c079d7: Status 404 returned error can't find the container with id b6720fcb95dabcc3816e1226be9f8d33a3e7e9b03d228299a7ec129633c079d7
Jul 31 23:50:31.371082 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:31.371043    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq" event={"ID":"091768bf-2dae-4e4f-aa46-52b2ae8c140d","Type":"ContainerStarted","Data":"b6720fcb95dabcc3816e1226be9f8d33a3e7e9b03d228299a7ec129633c079d7"}
Jul 31 23:50:36.390486 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:36.390447    2581 generic.go:356] "Generic (PLEG): container finished" podID="091768bf-2dae-4e4f-aa46-52b2ae8c140d" containerID="6b05a87f4ea31427c921676e4898bdf25ebc0069af230e522106d3e0ddda53ee" exitCode=0
Jul 31 23:50:36.390905 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:36.390529    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq" event={"ID":"091768bf-2dae-4e4f-aa46-52b2ae8c140d","Type":"ContainerDied","Data":"6b05a87f4ea31427c921676e4898bdf25ebc0069af230e522106d3e0ddda53ee"}
Jul 31 23:50:39.402276 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:39.402187    2581 generic.go:356] "Generic (PLEG): container finished" podID="091768bf-2dae-4e4f-aa46-52b2ae8c140d" containerID="f26d8eef0ae4a9bc13acd7b055fee152d596fbb2fc8d64e0ecc04feb706390d0" exitCode=0
Jul 31 23:50:39.402276 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:39.402264    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq" event={"ID":"091768bf-2dae-4e4f-aa46-52b2ae8c140d","Type":"ContainerDied","Data":"f26d8eef0ae4a9bc13acd7b055fee152d596fbb2fc8d64e0ecc04feb706390d0"}
Jul 31 23:50:45.425768 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:45.425678    2581 generic.go:356] "Generic (PLEG): container finished" podID="091768bf-2dae-4e4f-aa46-52b2ae8c140d" containerID="f3df1680868f08221babc403f9bfb78e4c664944a9016f296574a206a9ee5f93" exitCode=0
Jul 31 23:50:45.426179 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:45.425764    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq" event={"ID":"091768bf-2dae-4e4f-aa46-52b2ae8c140d","Type":"ContainerDied","Data":"f3df1680868f08221babc403f9bfb78e4c664944a9016f296574a206a9ee5f93"}
Jul 31 23:50:46.547883 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:46.547856    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq"
Jul 31 23:50:46.681517 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:46.681437    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/091768bf-2dae-4e4f-aa46-52b2ae8c140d-util\") pod \"091768bf-2dae-4e4f-aa46-52b2ae8c140d\" (UID: \"091768bf-2dae-4e4f-aa46-52b2ae8c140d\") "
Jul 31 23:50:46.681517 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:46.681503    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/091768bf-2dae-4e4f-aa46-52b2ae8c140d-bundle\") pod \"091768bf-2dae-4e4f-aa46-52b2ae8c140d\" (UID: \"091768bf-2dae-4e4f-aa46-52b2ae8c140d\") "
Jul 31 23:50:46.681696 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:46.681533    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5p7qv\" (UniqueName: \"kubernetes.io/projected/091768bf-2dae-4e4f-aa46-52b2ae8c140d-kube-api-access-5p7qv\") pod \"091768bf-2dae-4e4f-aa46-52b2ae8c140d\" (UID: \"091768bf-2dae-4e4f-aa46-52b2ae8c140d\") "
Jul 31 23:50:46.682165 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:46.682124    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/091768bf-2dae-4e4f-aa46-52b2ae8c140d-bundle" (OuterVolumeSpecName: "bundle") pod "091768bf-2dae-4e4f-aa46-52b2ae8c140d" (UID: "091768bf-2dae-4e4f-aa46-52b2ae8c140d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 23:50:46.683747 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:46.683716    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/091768bf-2dae-4e4f-aa46-52b2ae8c140d-kube-api-access-5p7qv" (OuterVolumeSpecName: "kube-api-access-5p7qv") pod "091768bf-2dae-4e4f-aa46-52b2ae8c140d" (UID: "091768bf-2dae-4e4f-aa46-52b2ae8c140d"). InnerVolumeSpecName "kube-api-access-5p7qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:50:46.685990 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:46.685967    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/091768bf-2dae-4e4f-aa46-52b2ae8c140d-util" (OuterVolumeSpecName: "util") pod "091768bf-2dae-4e4f-aa46-52b2ae8c140d" (UID: "091768bf-2dae-4e4f-aa46-52b2ae8c140d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 23:50:46.782895 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:46.782857    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/091768bf-2dae-4e4f-aa46-52b2ae8c140d-bundle\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:50:46.782895 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:46.782888    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p7qv\" (UniqueName: \"kubernetes.io/projected/091768bf-2dae-4e4f-aa46-52b2ae8c140d-kube-api-access-5p7qv\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:50:46.782895 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:46.782901    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/091768bf-2dae-4e4f-aa46-52b2ae8c140d-util\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:50:47.434550 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:47.434512    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq" event={"ID":"091768bf-2dae-4e4f-aa46-52b2ae8c140d","Type":"ContainerDied","Data":"b6720fcb95dabcc3816e1226be9f8d33a3e7e9b03d228299a7ec129633c079d7"}
Jul 31 23:50:47.434550 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:47.434532    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b576xkq"
Jul 31 23:50:47.434550 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:47.434561    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b6720fcb95dabcc3816e1226be9f8d33a3e7e9b03d228299a7ec129633c079d7"
Jul 31 23:50:57.180730 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.180699    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-77f648d694-nqtlk"]
Jul 31 23:50:57.181163 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.180915    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091768bf-2dae-4e4f-aa46-52b2ae8c140d" containerName="extract"
Jul 31 23:50:57.181163 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.180926    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="091768bf-2dae-4e4f-aa46-52b2ae8c140d" containerName="extract"
Jul 31 23:50:57.181163 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.180934    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091768bf-2dae-4e4f-aa46-52b2ae8c140d" containerName="util"
Jul 31 23:50:57.181163 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.180939    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="091768bf-2dae-4e4f-aa46-52b2ae8c140d" containerName="util"
Jul 31 23:50:57.181163 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.180947    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091768bf-2dae-4e4f-aa46-52b2ae8c140d" containerName="pull"
Jul 31 23:50:57.181163 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.180967    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="091768bf-2dae-4e4f-aa46-52b2ae8c140d" containerName="pull"
Jul 31 23:50:57.181163 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.181015    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="091768bf-2dae-4e4f-aa46-52b2ae8c140d" containerName="extract"
Jul 31 23:50:57.183929 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.183913    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-nqtlk"
Jul 31 23:50:57.186536 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.186512    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 31 23:50:57.186669 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.186620    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-hvqdd\""
Jul 31 23:50:57.187705 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.187684    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Jul 31 23:50:57.187705 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.187695    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 31 23:50:57.187891 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.187712    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 31 23:50:57.187891 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.187689    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 31 23:50:57.192204 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.192183    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-nqtlk"]
Jul 31 23:50:57.254920 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.254866    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/5211682e-58da-4a61-9b34-3fe0326ed2f8-certificates\") pod \"keda-operator-77f648d694-nqtlk\" (UID: \"5211682e-58da-4a61-9b34-3fe0326ed2f8\") " pod="openshift-keda/keda-operator-77f648d694-nqtlk"
Jul 31 23:50:57.254920 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.254926    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/5211682e-58da-4a61-9b34-3fe0326ed2f8-cabundle0\") pod \"keda-operator-77f648d694-nqtlk\" (UID: \"5211682e-58da-4a61-9b34-3fe0326ed2f8\") " pod="openshift-keda/keda-operator-77f648d694-nqtlk"
Jul 31 23:50:57.255166 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.254975    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqmbp\" (UniqueName: \"kubernetes.io/projected/5211682e-58da-4a61-9b34-3fe0326ed2f8-kube-api-access-hqmbp\") pod \"keda-operator-77f648d694-nqtlk\" (UID: \"5211682e-58da-4a61-9b34-3fe0326ed2f8\") " pod="openshift-keda/keda-operator-77f648d694-nqtlk"
Jul 31 23:50:57.356258 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.356227    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/5211682e-58da-4a61-9b34-3fe0326ed2f8-certificates\") pod \"keda-operator-77f648d694-nqtlk\" (UID: \"5211682e-58da-4a61-9b34-3fe0326ed2f8\") " pod="openshift-keda/keda-operator-77f648d694-nqtlk"
Jul 31 23:50:57.356440 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.356332    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/5211682e-58da-4a61-9b34-3fe0326ed2f8-cabundle0\") pod \"keda-operator-77f648d694-nqtlk\" (UID: \"5211682e-58da-4a61-9b34-3fe0326ed2f8\") " pod="openshift-keda/keda-operator-77f648d694-nqtlk"
Jul 31 23:50:57.356440 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:50:57.356348    2581 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Jul 31 23:50:57.356440 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.356357    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqmbp\" (UniqueName: \"kubernetes.io/projected/5211682e-58da-4a61-9b34-3fe0326ed2f8-kube-api-access-hqmbp\") pod \"keda-operator-77f648d694-nqtlk\" (UID: \"5211682e-58da-4a61-9b34-3fe0326ed2f8\") " pod="openshift-keda/keda-operator-77f648d694-nqtlk"
Jul 31 23:50:57.356440 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:50:57.356366    2581 secret.go:281] references non-existent secret key: ca.crt
Jul 31 23:50:57.356440 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:50:57.356373    2581 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 31 23:50:57.356440 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:50:57.356384    2581 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-nqtlk: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jul 31 23:50:57.356655 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:50:57.356457    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5211682e-58da-4a61-9b34-3fe0326ed2f8-certificates podName:5211682e-58da-4a61-9b34-3fe0326ed2f8 nodeName:}" failed. No retries permitted until 2026-07-31 23:50:57.85644037 +0000 UTC m=+281.035897091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/5211682e-58da-4a61-9b34-3fe0326ed2f8-certificates") pod "keda-operator-77f648d694-nqtlk" (UID: "5211682e-58da-4a61-9b34-3fe0326ed2f8") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jul 31 23:50:57.357073 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.357046    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/5211682e-58da-4a61-9b34-3fe0326ed2f8-cabundle0\") pod \"keda-operator-77f648d694-nqtlk\" (UID: \"5211682e-58da-4a61-9b34-3fe0326ed2f8\") " pod="openshift-keda/keda-operator-77f648d694-nqtlk"
Jul 31 23:50:57.365871 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.365842    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqmbp\" (UniqueName: \"kubernetes.io/projected/5211682e-58da-4a61-9b34-3fe0326ed2f8-kube-api-access-hqmbp\") pod \"keda-operator-77f648d694-nqtlk\" (UID: \"5211682e-58da-4a61-9b34-3fe0326ed2f8\") " pod="openshift-keda/keda-operator-77f648d694-nqtlk"
Jul 31 23:50:57.387014 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.386986    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"]
Jul 31 23:50:57.389815 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.389800    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"
Jul 31 23:50:57.392229 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.392205    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 31 23:50:57.398892 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.398864    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"]
Jul 31 23:50:57.457209 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.457116    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dxld\" (UniqueName: \"kubernetes.io/projected/efb2fa74-6a7d-41c0-ae09-02fd63763d78-kube-api-access-6dxld\") pod \"keda-metrics-apiserver-55557df8c-7bx7m\" (UID: \"efb2fa74-6a7d-41c0-ae09-02fd63763d78\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"
Jul 31 23:50:57.457209 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.457164    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/efb2fa74-6a7d-41c0-ae09-02fd63763d78-certificates\") pod \"keda-metrics-apiserver-55557df8c-7bx7m\" (UID: \"efb2fa74-6a7d-41c0-ae09-02fd63763d78\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"
Jul 31 23:50:57.457399 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.457212    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/efb2fa74-6a7d-41c0-ae09-02fd63763d78-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-7bx7m\" (UID: \"efb2fa74-6a7d-41c0-ae09-02fd63763d78\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"
Jul 31 23:50:57.557499 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.557458    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/efb2fa74-6a7d-41c0-ae09-02fd63763d78-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-7bx7m\" (UID: \"efb2fa74-6a7d-41c0-ae09-02fd63763d78\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"
Jul 31 23:50:57.557676 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.557512    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dxld\" (UniqueName: \"kubernetes.io/projected/efb2fa74-6a7d-41c0-ae09-02fd63763d78-kube-api-access-6dxld\") pod \"keda-metrics-apiserver-55557df8c-7bx7m\" (UID: \"efb2fa74-6a7d-41c0-ae09-02fd63763d78\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"
Jul 31 23:50:57.557676 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.557536    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/efb2fa74-6a7d-41c0-ae09-02fd63763d78-certificates\") pod \"keda-metrics-apiserver-55557df8c-7bx7m\" (UID: \"efb2fa74-6a7d-41c0-ae09-02fd63763d78\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"
Jul 31 23:50:57.557676 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:50:57.557626    2581 secret.go:281] references non-existent secret key: tls.crt
Jul 31 23:50:57.557676 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:50:57.557639    2581 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jul 31 23:50:57.557676 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:50:57.557653    2581 projected.go:266] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found
Jul 31 23:50:57.557676 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:50:57.557671    2581 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m: [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found]
Jul 31 23:50:57.557889 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:50:57.557719    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efb2fa74-6a7d-41c0-ae09-02fd63763d78-certificates podName:efb2fa74-6a7d-41c0-ae09-02fd63763d78 nodeName:}" failed. No retries permitted until 2026-07-31 23:50:58.05770558 +0000 UTC m=+281.237162303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/efb2fa74-6a7d-41c0-ae09-02fd63763d78-certificates") pod "keda-metrics-apiserver-55557df8c-7bx7m" (UID: "efb2fa74-6a7d-41c0-ae09-02fd63763d78") : [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found]
Jul 31 23:50:57.557934 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.557893    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/efb2fa74-6a7d-41c0-ae09-02fd63763d78-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-7bx7m\" (UID: \"efb2fa74-6a7d-41c0-ae09-02fd63763d78\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"
Jul 31 23:50:57.568347 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.568318    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dxld\" (UniqueName: \"kubernetes.io/projected/efb2fa74-6a7d-41c0-ae09-02fd63763d78-kube-api-access-6dxld\") pod \"keda-metrics-apiserver-55557df8c-7bx7m\" (UID: \"efb2fa74-6a7d-41c0-ae09-02fd63763d78\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"
Jul 31 23:50:57.858697 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:57.858597    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/5211682e-58da-4a61-9b34-3fe0326ed2f8-certificates\") pod \"keda-operator-77f648d694-nqtlk\" (UID: \"5211682e-58da-4a61-9b34-3fe0326ed2f8\") " pod="openshift-keda/keda-operator-77f648d694-nqtlk"
Jul 31 23:50:57.858862 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:50:57.858733    2581 secret.go:281] references non-existent secret key: ca.crt
Jul 31 23:50:57.858862 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:50:57.858750    2581 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 31 23:50:57.858862 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:50:57.858758    2581 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-nqtlk: references non-existent secret key: ca.crt
Jul 31 23:50:57.858862 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:50:57.858808    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5211682e-58da-4a61-9b34-3fe0326ed2f8-certificates podName:5211682e-58da-4a61-9b34-3fe0326ed2f8 nodeName:}" failed. No retries permitted until 2026-07-31 23:50:58.858792033 +0000 UTC m=+282.038248774 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/5211682e-58da-4a61-9b34-3fe0326ed2f8-certificates") pod "keda-operator-77f648d694-nqtlk" (UID: "5211682e-58da-4a61-9b34-3fe0326ed2f8") : references non-existent secret key: ca.crt
Jul 31 23:50:58.059702 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:58.059662    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/efb2fa74-6a7d-41c0-ae09-02fd63763d78-certificates\") pod \"keda-metrics-apiserver-55557df8c-7bx7m\" (UID: \"efb2fa74-6a7d-41c0-ae09-02fd63763d78\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"
Jul 31 23:50:58.062140 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:58.062118    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/efb2fa74-6a7d-41c0-ae09-02fd63763d78-certificates\") pod \"keda-metrics-apiserver-55557df8c-7bx7m\" (UID: \"efb2fa74-6a7d-41c0-ae09-02fd63763d78\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"
Jul 31 23:50:58.300079 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:58.300046    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"
Jul 31 23:50:58.424227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:58.424188    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"]
Jul 31 23:50:58.427633 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:50:58.427590    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb2fa74_6a7d_41c0_ae09_02fd63763d78.slice/crio-31c777127b1c6db9faeda21055a4a10718774f1d46a13cc88f0dbd5c7c6e90cd WatchSource:0}: Error finding container 31c777127b1c6db9faeda21055a4a10718774f1d46a13cc88f0dbd5c7c6e90cd: Status 404 returned error can't find the container with id 31c777127b1c6db9faeda21055a4a10718774f1d46a13cc88f0dbd5c7c6e90cd
Jul 31 23:50:58.469316 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:58.469272    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m" event={"ID":"efb2fa74-6a7d-41c0-ae09-02fd63763d78","Type":"ContainerStarted","Data":"31c777127b1c6db9faeda21055a4a10718774f1d46a13cc88f0dbd5c7c6e90cd"}
Jul 31 23:50:58.864723 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:58.864685    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/5211682e-58da-4a61-9b34-3fe0326ed2f8-certificates\") pod \"keda-operator-77f648d694-nqtlk\" (UID: \"5211682e-58da-4a61-9b34-3fe0326ed2f8\") " pod="openshift-keda/keda-operator-77f648d694-nqtlk"
Jul 31 23:50:58.867089 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:58.867067    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/5211682e-58da-4a61-9b34-3fe0326ed2f8-certificates\") pod \"keda-operator-77f648d694-nqtlk\" (UID: \"5211682e-58da-4a61-9b34-3fe0326ed2f8\") " pod="openshift-keda/keda-operator-77f648d694-nqtlk"
Jul 31 23:50:58.994294 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:58.994255    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-nqtlk"
Jul 31 23:50:59.126971 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:59.126922    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-nqtlk"]
Jul 31 23:50:59.130884 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:50:59.130857    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5211682e_58da_4a61_9b34_3fe0326ed2f8.slice/crio-dfd35918ad947bdf2624f0d6fd0ab295c00bb6e23ab3a12e7479acbdfb26da55 WatchSource:0}: Error finding container dfd35918ad947bdf2624f0d6fd0ab295c00bb6e23ab3a12e7479acbdfb26da55: Status 404 returned error can't find the container with id dfd35918ad947bdf2624f0d6fd0ab295c00bb6e23ab3a12e7479acbdfb26da55
Jul 31 23:50:59.473820 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:50:59.473742    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-nqtlk" event={"ID":"5211682e-58da-4a61-9b34-3fe0326ed2f8","Type":"ContainerStarted","Data":"dfd35918ad947bdf2624f0d6fd0ab295c00bb6e23ab3a12e7479acbdfb26da55"}
Jul 31 23:51:01.484707 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:51:01.484655    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m" event={"ID":"efb2fa74-6a7d-41c0-ae09-02fd63763d78","Type":"ContainerStarted","Data":"75d04cc6a4a3e184d98fafd9ac66c24af3097d98931fc7c5c00682a80d00f9ce"}
Jul 31 23:51:01.485238 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:51:01.484990    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"
Jul 31 23:51:01.503028 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:51:01.502947    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m" podStartSLOduration=1.5778495860000001 podStartE2EDuration="4.502924399s" totalImagesPullingTime="2.925074813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:50:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:51:01.500667268 +0000 UTC m=+284.680124013" watchObservedRunningTime="2026-07-31 23:51:01.502924399 +0000 UTC m=+284.682381143"
Jul 31 23:51:03.492976 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:51:03.492921    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-nqtlk" event={"ID":"5211682e-58da-4a61-9b34-3fe0326ed2f8","Type":"ContainerStarted","Data":"bb16c084d3d80ba13425a8b56adcdf12b930d89529eeb6e1f7aa8bd9860df505"}
Jul 31 23:51:03.493390 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:51:03.493086    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-77f648d694-nqtlk"
Jul 31 23:51:03.509061 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:51:03.509006    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-77f648d694-nqtlk" podStartSLOduration=2.9588900259999997 podStartE2EDuration="6.50899031s" totalImagesPullingTime="3.550100284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:50:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:51:03.50881031 +0000 UTC m=+286.688267058" watchObservedRunningTime="2026-07-31 23:51:03.50899031 +0000 UTC m=+286.688447050"
Jul 31 23:51:12.493060 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:51:12.493029    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-7bx7m"
Jul 31 23:51:17.256235 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:51:17.256194    2581 kubelet.go:1648] "Image garbage collection succeeded"
Jul 31 23:51:24.498649 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:51:24.498610    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-77f648d694-nqtlk"
Jul 31 23:52:15.401187 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:15.401098    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-kcgr2"]
Jul 31 23:52:15.404258 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:15.404240    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-654c557856-kcgr2"
Jul 31 23:52:15.407890 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:15.407860    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 31 23:52:15.408129 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:15.408111    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 31 23:52:15.408547 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:15.408522    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-6xlj5\""
Jul 31 23:52:15.408664 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:15.408550    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jul 31 23:52:15.416107 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:15.416023    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-kcgr2"]
Jul 31 23:52:15.527815 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:15.527775    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbcpt\" (UniqueName: \"kubernetes.io/projected/bbe6029c-9520-421d-b0ca-d6b283c65a20-kube-api-access-dbcpt\") pod \"llmisvc-controller-manager-654c557856-kcgr2\" (UID: \"bbe6029c-9520-421d-b0ca-d6b283c65a20\") " pod="kserve/llmisvc-controller-manager-654c557856-kcgr2"
Jul 31 23:52:15.528046 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:15.527835    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbe6029c-9520-421d-b0ca-d6b283c65a20-cert\") pod \"llmisvc-controller-manager-654c557856-kcgr2\" (UID: \"bbe6029c-9520-421d-b0ca-d6b283c65a20\") " pod="kserve/llmisvc-controller-manager-654c557856-kcgr2"
Jul 31 23:52:15.628385 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:15.628344    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbcpt\" (UniqueName: \"kubernetes.io/projected/bbe6029c-9520-421d-b0ca-d6b283c65a20-kube-api-access-dbcpt\") pod \"llmisvc-controller-manager-654c557856-kcgr2\" (UID: \"bbe6029c-9520-421d-b0ca-d6b283c65a20\") " pod="kserve/llmisvc-controller-manager-654c557856-kcgr2"
Jul 31 23:52:15.628584 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:15.628412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbe6029c-9520-421d-b0ca-d6b283c65a20-cert\") pod \"llmisvc-controller-manager-654c557856-kcgr2\" (UID: \"bbe6029c-9520-421d-b0ca-d6b283c65a20\") " pod="kserve/llmisvc-controller-manager-654c557856-kcgr2"
Jul 31 23:52:15.628584 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:52:15.628558    2581 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 31 23:52:15.628675 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:52:15.628623    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbe6029c-9520-421d-b0ca-d6b283c65a20-cert podName:bbe6029c-9520-421d-b0ca-d6b283c65a20 nodeName:}" failed. No retries permitted until 2026-07-31 23:52:16.128606022 +0000 UTC m=+359.308062749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bbe6029c-9520-421d-b0ca-d6b283c65a20-cert") pod "llmisvc-controller-manager-654c557856-kcgr2" (UID: "bbe6029c-9520-421d-b0ca-d6b283c65a20") : secret "llmisvc-webhook-server-cert" not found
Jul 31 23:52:15.639045 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:15.639018    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbcpt\" (UniqueName: \"kubernetes.io/projected/bbe6029c-9520-421d-b0ca-d6b283c65a20-kube-api-access-dbcpt\") pod \"llmisvc-controller-manager-654c557856-kcgr2\" (UID: \"bbe6029c-9520-421d-b0ca-d6b283c65a20\") " pod="kserve/llmisvc-controller-manager-654c557856-kcgr2"
Jul 31 23:52:16.130926 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:16.130869    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbe6029c-9520-421d-b0ca-d6b283c65a20-cert\") pod \"llmisvc-controller-manager-654c557856-kcgr2\" (UID: \"bbe6029c-9520-421d-b0ca-d6b283c65a20\") " pod="kserve/llmisvc-controller-manager-654c557856-kcgr2"
Jul 31 23:52:16.133371 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:16.133341    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbe6029c-9520-421d-b0ca-d6b283c65a20-cert\") pod \"llmisvc-controller-manager-654c557856-kcgr2\" (UID: \"bbe6029c-9520-421d-b0ca-d6b283c65a20\") " pod="kserve/llmisvc-controller-manager-654c557856-kcgr2"
Jul 31 23:52:16.314569 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:16.314515    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-654c557856-kcgr2"
Jul 31 23:52:16.449924 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:16.449892    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-kcgr2"]
Jul 31 23:52:16.452604 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:52:16.452571    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbbe6029c_9520_421d_b0ca_d6b283c65a20.slice/crio-d50f5288c5317bb99e2b433d47bc1f7b0cfc79c9c761527f17a3a622b27b200d WatchSource:0}: Error finding container d50f5288c5317bb99e2b433d47bc1f7b0cfc79c9c761527f17a3a622b27b200d: Status 404 returned error can't find the container with id d50f5288c5317bb99e2b433d47bc1f7b0cfc79c9c761527f17a3a622b27b200d
Jul 31 23:52:16.454187 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:16.454166    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 23:52:16.733433 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:16.733344    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-kcgr2" event={"ID":"bbe6029c-9520-421d-b0ca-d6b283c65a20","Type":"ContainerStarted","Data":"d50f5288c5317bb99e2b433d47bc1f7b0cfc79c9c761527f17a3a622b27b200d"}
Jul 31 23:52:18.742475 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:18.742437    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-kcgr2" event={"ID":"bbe6029c-9520-421d-b0ca-d6b283c65a20","Type":"ContainerStarted","Data":"66e28e79f6b1f21cf34c212476c581990d1efdee5e97f1dc93c4f4e0ebd19433"}
Jul 31 23:52:18.742980 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:18.742535    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-654c557856-kcgr2"
Jul 31 23:52:32.124420 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.124367    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-654c557856-kcgr2" podStartSLOduration=15.216663926 podStartE2EDuration="17.124349707s" totalImagesPullingTime="1.907685781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:52:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:52:18.757867464 +0000 UTC m=+361.937324208" watchObservedRunningTime="2026-07-31 23:52:32.124349707 +0000 UTC m=+375.303806454"
Jul 31 23:52:32.125188 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.125172    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-645997df77-hps8r"]
Jul 31 23:52:32.128346 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.128322    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.137655 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.137626    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-645997df77-hps8r"]
Jul 31 23:52:32.149189 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.149158    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a9f0928-d041-4f5e-9539-8281ea50ecb3-service-ca\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.149355 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.149199    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fbdn\" (UniqueName: \"kubernetes.io/projected/2a9f0928-d041-4f5e-9539-8281ea50ecb3-kube-api-access-4fbdn\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.149355 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.149273    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a9f0928-d041-4f5e-9539-8281ea50ecb3-oauth-serving-cert\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.149355 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.149316    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a9f0928-d041-4f5e-9539-8281ea50ecb3-console-serving-cert\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.149493 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.149354    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a9f0928-d041-4f5e-9539-8281ea50ecb3-trusted-ca-bundle\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.149493 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.149392    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a9f0928-d041-4f5e-9539-8281ea50ecb3-console-oauth-config\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.149493 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.149433    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a9f0928-d041-4f5e-9539-8281ea50ecb3-console-config\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.250227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.250196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a9f0928-d041-4f5e-9539-8281ea50ecb3-service-ca\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.250227 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.250234    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fbdn\" (UniqueName: \"kubernetes.io/projected/2a9f0928-d041-4f5e-9539-8281ea50ecb3-kube-api-access-4fbdn\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.250485 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.250265    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a9f0928-d041-4f5e-9539-8281ea50ecb3-oauth-serving-cert\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.250485 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.250287    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a9f0928-d041-4f5e-9539-8281ea50ecb3-console-serving-cert\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.250485 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.250315    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a9f0928-d041-4f5e-9539-8281ea50ecb3-trusted-ca-bundle\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.250485 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.250345    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a9f0928-d041-4f5e-9539-8281ea50ecb3-console-oauth-config\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.250485 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.250385    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a9f0928-d041-4f5e-9539-8281ea50ecb3-console-config\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.251038 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.251013    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a9f0928-d041-4f5e-9539-8281ea50ecb3-service-ca\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.251038 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.251025    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a9f0928-d041-4f5e-9539-8281ea50ecb3-oauth-serving-cert\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.251173 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.251129    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a9f0928-d041-4f5e-9539-8281ea50ecb3-console-config\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.251427 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.251401    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a9f0928-d041-4f5e-9539-8281ea50ecb3-trusted-ca-bundle\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.252782 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.252751    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a9f0928-d041-4f5e-9539-8281ea50ecb3-console-oauth-config\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.252936 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.252917    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a9f0928-d041-4f5e-9539-8281ea50ecb3-console-serving-cert\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.258479 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.258460    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fbdn\" (UniqueName: \"kubernetes.io/projected/2a9f0928-d041-4f5e-9539-8281ea50ecb3-kube-api-access-4fbdn\") pod \"console-645997df77-hps8r\" (UID: \"2a9f0928-d041-4f5e-9539-8281ea50ecb3\") " pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.437762 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.437724    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:32.567336 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.567301    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-645997df77-hps8r"]
Jul 31 23:52:32.570232 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:52:32.570202    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a9f0928_d041_4f5e_9539_8281ea50ecb3.slice/crio-fe29da383bbb64ea120b6b2afad3900ce2ba2c15de5481c623ebe6e4b0756bfe WatchSource:0}: Error finding container fe29da383bbb64ea120b6b2afad3900ce2ba2c15de5481c623ebe6e4b0756bfe: Status 404 returned error can't find the container with id fe29da383bbb64ea120b6b2afad3900ce2ba2c15de5481c623ebe6e4b0756bfe
Jul 31 23:52:32.790998 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.790891    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645997df77-hps8r" event={"ID":"2a9f0928-d041-4f5e-9539-8281ea50ecb3","Type":"ContainerStarted","Data":"6991b677349f48b2254dbf417b0055c579ef904e465d41ed98a488a213ad897d"}
Jul 31 23:52:32.790998 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.790947    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645997df77-hps8r" event={"ID":"2a9f0928-d041-4f5e-9539-8281ea50ecb3","Type":"ContainerStarted","Data":"fe29da383bbb64ea120b6b2afad3900ce2ba2c15de5481c623ebe6e4b0756bfe"}
Jul 31 23:52:32.808728 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:32.808672    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-645997df77-hps8r" podStartSLOduration=0.808654771 podStartE2EDuration="808.654771ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:52:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:52:32.807890527 +0000 UTC m=+375.987347270" watchObservedRunningTime="2026-07-31 23:52:32.808654771 +0000 UTC m=+375.988111558"
Jul 31 23:52:42.438820 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:42.438779    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:42.439369 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:42.438843    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:42.443905 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:42.443879    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:42.829379 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:42.829290    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-645997df77-hps8r"
Jul 31 23:52:42.885641 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:42.885606    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67d7b7779-n5rt6"]
Jul 31 23:52:49.749389 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:52:49.749352    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-654c557856-kcgr2"
Jul 31 23:53:07.905836 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:07.905793    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67d7b7779-n5rt6" podUID="e44df733-fe2b-459b-a84e-a27acca6e2cf" containerName="console" containerID="cri-o://2293eaa72ace470910682d1c8baf21ef9a9b6bedd869173c9d996f402e2f1e3c" gracePeriod=15
Jul 31 23:53:08.146319 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.146292    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:53:08.225695 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.225600    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-service-ca\") pod \"e44df733-fe2b-459b-a84e-a27acca6e2cf\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") "
Jul 31 23:53:08.225695 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.225648    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-serving-cert\") pod \"e44df733-fe2b-459b-a84e-a27acca6e2cf\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") "
Jul 31 23:53:08.225695 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.225677    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-oauth-serving-cert\") pod \"e44df733-fe2b-459b-a84e-a27acca6e2cf\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") "
Jul 31 23:53:08.225695 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.225696    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-config\") pod \"e44df733-fe2b-459b-a84e-a27acca6e2cf\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") "
Jul 31 23:53:08.225983 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.225744    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-trusted-ca-bundle\") pod \"e44df733-fe2b-459b-a84e-a27acca6e2cf\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") "
Jul 31 23:53:08.225983 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.225804    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-oauth-config\") pod \"e44df733-fe2b-459b-a84e-a27acca6e2cf\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") "
Jul 31 23:53:08.225983 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.225830    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdhdd\" (UniqueName: \"kubernetes.io/projected/e44df733-fe2b-459b-a84e-a27acca6e2cf-kube-api-access-zdhdd\") pod \"e44df733-fe2b-459b-a84e-a27acca6e2cf\" (UID: \"e44df733-fe2b-459b-a84e-a27acca6e2cf\") "
Jul 31 23:53:08.226169 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.226127    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e44df733-fe2b-459b-a84e-a27acca6e2cf" (UID: "e44df733-fe2b-459b-a84e-a27acca6e2cf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:53:08.226320 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.226294    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e44df733-fe2b-459b-a84e-a27acca6e2cf" (UID: "e44df733-fe2b-459b-a84e-a27acca6e2cf"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:53:08.226415 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.226310    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-config" (OuterVolumeSpecName: "console-config") pod "e44df733-fe2b-459b-a84e-a27acca6e2cf" (UID: "e44df733-fe2b-459b-a84e-a27acca6e2cf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:53:08.226415 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.226373    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-service-ca" (OuterVolumeSpecName: "service-ca") pod "e44df733-fe2b-459b-a84e-a27acca6e2cf" (UID: "e44df733-fe2b-459b-a84e-a27acca6e2cf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:53:08.228068 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.228044    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e44df733-fe2b-459b-a84e-a27acca6e2cf-kube-api-access-zdhdd" (OuterVolumeSpecName: "kube-api-access-zdhdd") pod "e44df733-fe2b-459b-a84e-a27acca6e2cf" (UID: "e44df733-fe2b-459b-a84e-a27acca6e2cf"). InnerVolumeSpecName "kube-api-access-zdhdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:53:08.228353 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.228328    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e44df733-fe2b-459b-a84e-a27acca6e2cf" (UID: "e44df733-fe2b-459b-a84e-a27acca6e2cf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:53:08.228410 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.228351    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e44df733-fe2b-459b-a84e-a27acca6e2cf" (UID: "e44df733-fe2b-459b-a84e-a27acca6e2cf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:53:08.326518 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.326479    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-trusted-ca-bundle\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:53:08.326518 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.326510    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-oauth-config\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:53:08.326518 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.326519    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdhdd\" (UniqueName: \"kubernetes.io/projected/e44df733-fe2b-459b-a84e-a27acca6e2cf-kube-api-access-zdhdd\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:53:08.326518 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.326531    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-service-ca\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:53:08.326800 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.326539    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-serving-cert\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:53:08.326800 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.326548    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-oauth-serving-cert\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:53:08.326800 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.326556    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e44df733-fe2b-459b-a84e-a27acca6e2cf-console-config\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:53:08.916258 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.916218    2581 generic.go:356] "Generic (PLEG): container finished" podID="e44df733-fe2b-459b-a84e-a27acca6e2cf" containerID="2293eaa72ace470910682d1c8baf21ef9a9b6bedd869173c9d996f402e2f1e3c" exitCode=0
Jul 31 23:53:08.916258 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.916264    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d7b7779-n5rt6" event={"ID":"e44df733-fe2b-459b-a84e-a27acca6e2cf","Type":"ContainerDied","Data":"2293eaa72ace470910682d1c8baf21ef9a9b6bedd869173c9d996f402e2f1e3c"}
Jul 31 23:53:08.916725 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.916304    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d7b7779-n5rt6" event={"ID":"e44df733-fe2b-459b-a84e-a27acca6e2cf","Type":"ContainerDied","Data":"6bbae17bbea3900ca132b3e4209b53ef11155f4d79215a4748ef0a7d3211939d"}
Jul 31 23:53:08.916725 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.916324    2581 scope.go:117] "RemoveContainer" containerID="2293eaa72ace470910682d1c8baf21ef9a9b6bedd869173c9d996f402e2f1e3c"
Jul 31 23:53:08.916725 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.916266    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d7b7779-n5rt6"
Jul 31 23:53:08.924680 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.924657    2581 scope.go:117] "RemoveContainer" containerID="2293eaa72ace470910682d1c8baf21ef9a9b6bedd869173c9d996f402e2f1e3c"
Jul 31 23:53:08.924990 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:53:08.924947    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2293eaa72ace470910682d1c8baf21ef9a9b6bedd869173c9d996f402e2f1e3c\": container with ID starting with 2293eaa72ace470910682d1c8baf21ef9a9b6bedd869173c9d996f402e2f1e3c not found: ID does not exist" containerID="2293eaa72ace470910682d1c8baf21ef9a9b6bedd869173c9d996f402e2f1e3c"
Jul 31 23:53:08.925067 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.925000    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2293eaa72ace470910682d1c8baf21ef9a9b6bedd869173c9d996f402e2f1e3c"} err="failed to get container status \"2293eaa72ace470910682d1c8baf21ef9a9b6bedd869173c9d996f402e2f1e3c\": rpc error: code = NotFound desc = could not find container \"2293eaa72ace470910682d1c8baf21ef9a9b6bedd869173c9d996f402e2f1e3c\": container with ID starting with 2293eaa72ace470910682d1c8baf21ef9a9b6bedd869173c9d996f402e2f1e3c not found: ID does not exist"
Jul 31 23:53:08.936864 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.936828    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67d7b7779-n5rt6"]
Jul 31 23:53:08.938714 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:08.938691    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67d7b7779-n5rt6"]
Jul 31 23:53:09.386932 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:09.386889    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e44df733-fe2b-459b-a84e-a27acca6e2cf" path="/var/lib/kubelet/pods/e44df733-fe2b-459b-a84e-a27acca6e2cf/volumes"
Jul 31 23:53:30.548550 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:30.548519    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-7b954ffc95-xvsbm"]
Jul 31 23:53:30.549002 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:30.548760    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e44df733-fe2b-459b-a84e-a27acca6e2cf" containerName="console"
Jul 31 23:53:30.549002 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:30.548770    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e44df733-fe2b-459b-a84e-a27acca6e2cf" containerName="console"
Jul 31 23:53:30.549002 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:30.548829    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e44df733-fe2b-459b-a84e-a27acca6e2cf" containerName="console"
Jul 31 23:53:30.552713 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:30.552697    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-xvsbm"
Jul 31 23:53:30.554986 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:30.554948    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Jul 31 23:53:30.555116 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:30.555030    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-k5khc\""
Jul 31 23:53:30.564941 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:30.564914    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-xvsbm"]
Jul 31 23:53:30.696084 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:30.696048    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d19a962-96c7-406c-8a33-42b17e748ab4-tls-certs\") pod \"model-serving-api-7b954ffc95-xvsbm\" (UID: \"8d19a962-96c7-406c-8a33-42b17e748ab4\") " pod="kserve/model-serving-api-7b954ffc95-xvsbm"
Jul 31 23:53:30.696253 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:30.696102    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs2fm\" (UniqueName: \"kubernetes.io/projected/8d19a962-96c7-406c-8a33-42b17e748ab4-kube-api-access-qs2fm\") pod \"model-serving-api-7b954ffc95-xvsbm\" (UID: \"8d19a962-96c7-406c-8a33-42b17e748ab4\") " pod="kserve/model-serving-api-7b954ffc95-xvsbm"
Jul 31 23:53:30.797086 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:30.797053    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d19a962-96c7-406c-8a33-42b17e748ab4-tls-certs\") pod \"model-serving-api-7b954ffc95-xvsbm\" (UID: \"8d19a962-96c7-406c-8a33-42b17e748ab4\") " pod="kserve/model-serving-api-7b954ffc95-xvsbm"
Jul 31 23:53:30.797208 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:30.797102    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs2fm\" (UniqueName: \"kubernetes.io/projected/8d19a962-96c7-406c-8a33-42b17e748ab4-kube-api-access-qs2fm\") pod \"model-serving-api-7b954ffc95-xvsbm\" (UID: \"8d19a962-96c7-406c-8a33-42b17e748ab4\") " pod="kserve/model-serving-api-7b954ffc95-xvsbm"
Jul 31 23:53:30.797245 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:53:30.797214    2581 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 31 23:53:30.797297 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:53:30.797285    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d19a962-96c7-406c-8a33-42b17e748ab4-tls-certs podName:8d19a962-96c7-406c-8a33-42b17e748ab4 nodeName:}" failed. No retries permitted until 2026-07-31 23:53:31.297267736 +0000 UTC m=+434.476724459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8d19a962-96c7-406c-8a33-42b17e748ab4-tls-certs") pod "model-serving-api-7b954ffc95-xvsbm" (UID: "8d19a962-96c7-406c-8a33-42b17e748ab4") : secret "model-serving-api-tls" not found
Jul 31 23:53:30.805782 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:30.805711    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs2fm\" (UniqueName: \"kubernetes.io/projected/8d19a962-96c7-406c-8a33-42b17e748ab4-kube-api-access-qs2fm\") pod \"model-serving-api-7b954ffc95-xvsbm\" (UID: \"8d19a962-96c7-406c-8a33-42b17e748ab4\") " pod="kserve/model-serving-api-7b954ffc95-xvsbm"
Jul 31 23:53:31.301538 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:31.301502    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d19a962-96c7-406c-8a33-42b17e748ab4-tls-certs\") pod \"model-serving-api-7b954ffc95-xvsbm\" (UID: \"8d19a962-96c7-406c-8a33-42b17e748ab4\") " pod="kserve/model-serving-api-7b954ffc95-xvsbm"
Jul 31 23:53:31.303910 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:31.303890    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d19a962-96c7-406c-8a33-42b17e748ab4-tls-certs\") pod \"model-serving-api-7b954ffc95-xvsbm\" (UID: \"8d19a962-96c7-406c-8a33-42b17e748ab4\") " pod="kserve/model-serving-api-7b954ffc95-xvsbm"
Jul 31 23:53:31.463087 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:31.463046    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-xvsbm"
Jul 31 23:53:31.590939 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:31.590847    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-xvsbm"]
Jul 31 23:53:31.595852 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:53:31.595819    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d19a962_96c7_406c_8a33_42b17e748ab4.slice/crio-285ddfbe9a0f99050d9ab3eff1f6ca249f51c1239e70ade2c757008436f84747 WatchSource:0}: Error finding container 285ddfbe9a0f99050d9ab3eff1f6ca249f51c1239e70ade2c757008436f84747: Status 404 returned error can't find the container with id 285ddfbe9a0f99050d9ab3eff1f6ca249f51c1239e70ade2c757008436f84747
Jul 31 23:53:31.996625 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:31.996587    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-xvsbm" event={"ID":"8d19a962-96c7-406c-8a33-42b17e748ab4","Type":"ContainerStarted","Data":"285ddfbe9a0f99050d9ab3eff1f6ca249f51c1239e70ade2c757008436f84747"}
Jul 31 23:53:34.005020 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:34.004930    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-xvsbm" event={"ID":"8d19a962-96c7-406c-8a33-42b17e748ab4","Type":"ContainerStarted","Data":"ce140f3bf5799ab0fe2e905ca263c1839eff39b2bcb76f436173a8bbe171c996"}
Jul 31 23:53:34.005388 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:34.005059    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-7b954ffc95-xvsbm"
Jul 31 23:53:34.021804 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:34.021751    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-7b954ffc95-xvsbm" podStartSLOduration=1.90323453 podStartE2EDuration="4.021732481s" totalImagesPullingTime="2.118497951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:53:34.020746342 +0000 UTC m=+437.200203086" watchObservedRunningTime="2026-07-31 23:53:34.021732481 +0000 UTC m=+437.201189225"
Jul 31 23:53:45.016669 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:53:45.016593    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-7b954ffc95-xvsbm"
Jul 31 23:55:15.155886 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.155806    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"]
Jul 31 23:55:15.159097 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.159081    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:15.161689 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.161666    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 31 23:55:15.161830 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.161807    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-raw-sklearn-batcher-76a60-kube-rbac-proxy-sar-config\""
Jul 31 23:55:15.161884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.161826    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 31 23:55:15.161884 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.161844    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-raw-sklearn-batcher-76a60-predictor-serving-cert\""
Jul 31 23:55:15.161978 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.161928    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 31 23:55:15.162078 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.162065    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wctjk\""
Jul 31 23:55:15.162171 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.162156    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 31 23:55:15.169189 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.169164    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"]
Jul 31 23:55:15.173850 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.173829    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56e7c07e-0328-4d43-84e6-3017ac123d75-kserve-provision-location\") pod \"isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:15.173948 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.173865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56e7c07e-0328-4d43-84e6-3017ac123d75-kube-rbac-proxy-sar-config\") pod \"isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:15.173948 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.173886    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56e7c07e-0328-4d43-84e6-3017ac123d75-cabundle-cert\") pod \"isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:15.174052 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.174002    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56e7c07e-0328-4d43-84e6-3017ac123d75-proxy-tls\") pod \"isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:15.174052 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.174033    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm62z\" (UniqueName: \"kubernetes.io/projected/56e7c07e-0328-4d43-84e6-3017ac123d75-kube-api-access-lm62z\") pod \"isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:15.275352 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.275318    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56e7c07e-0328-4d43-84e6-3017ac123d75-proxy-tls\") pod \"isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:15.275541 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.275359    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm62z\" (UniqueName: \"kubernetes.io/projected/56e7c07e-0328-4d43-84e6-3017ac123d75-kube-api-access-lm62z\") pod \"isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:15.275541 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.275459    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56e7c07e-0328-4d43-84e6-3017ac123d75-kserve-provision-location\") pod \"isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:15.275541 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.275498    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56e7c07e-0328-4d43-84e6-3017ac123d75-kube-rbac-proxy-sar-config\") pod \"isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:15.275541 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:55:15.275506    2581 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-serving-cert: secret "isvc-raw-sklearn-batcher-76a60-predictor-serving-cert" not found
Jul 31 23:55:15.275761 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:55:15.275591    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56e7c07e-0328-4d43-84e6-3017ac123d75-proxy-tls podName:56e7c07e-0328-4d43-84e6-3017ac123d75 nodeName:}" failed. No retries permitted until 2026-07-31 23:55:15.775568822 +0000 UTC m=+538.955025547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/56e7c07e-0328-4d43-84e6-3017ac123d75-proxy-tls") pod "isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" (UID: "56e7c07e-0328-4d43-84e6-3017ac123d75") : secret "isvc-raw-sklearn-batcher-76a60-predictor-serving-cert" not found
Jul 31 23:55:15.275761 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.275615    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56e7c07e-0328-4d43-84e6-3017ac123d75-cabundle-cert\") pod \"isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:15.275936 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.275911    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56e7c07e-0328-4d43-84e6-3017ac123d75-kserve-provision-location\") pod \"isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:15.276183 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.276163    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56e7c07e-0328-4d43-84e6-3017ac123d75-kube-rbac-proxy-sar-config\") pod \"isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:15.276290 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.276254    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56e7c07e-0328-4d43-84e6-3017ac123d75-cabundle-cert\") pod \"isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:15.286315 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.286281    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm62z\" (UniqueName: \"kubernetes.io/projected/56e7c07e-0328-4d43-84e6-3017ac123d75-kube-api-access-lm62z\") pod \"isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:15.781432 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.781393    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56e7c07e-0328-4d43-84e6-3017ac123d75-proxy-tls\") pod \"isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:15.783881 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:15.783853    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56e7c07e-0328-4d43-84e6-3017ac123d75-proxy-tls\") pod \"isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:16.069321 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:16.069211    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:16.202491 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:16.202457    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"]
Jul 31 23:55:16.206521 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:55:16.206488    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56e7c07e_0328_4d43_84e6_3017ac123d75.slice/crio-16b66d877dc2cd60a5cdd69fac91143d892b44d07d2dc670ee817b00435dab37 WatchSource:0}: Error finding container 16b66d877dc2cd60a5cdd69fac91143d892b44d07d2dc670ee817b00435dab37: Status 404 returned error can't find the container with id 16b66d877dc2cd60a5cdd69fac91143d892b44d07d2dc670ee817b00435dab37
Jul 31 23:55:16.368939 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:16.368845    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" event={"ID":"56e7c07e-0328-4d43-84e6-3017ac123d75","Type":"ContainerStarted","Data":"16b66d877dc2cd60a5cdd69fac91143d892b44d07d2dc670ee817b00435dab37"}
Jul 31 23:55:19.024345 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:19.024320    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 31 23:55:19.381994 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:19.381936    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" event={"ID":"56e7c07e-0328-4d43-84e6-3017ac123d75","Type":"ContainerStarted","Data":"43dec387a40ef7ab2e3ce14b182d05e669d88f4bd7def60d492e91173fbed595"}
Jul 31 23:55:23.397475 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:23.397437    2581 generic.go:356] "Generic (PLEG): container finished" podID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerID="43dec387a40ef7ab2e3ce14b182d05e669d88f4bd7def60d492e91173fbed595" exitCode=0
Jul 31 23:55:23.397871 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:23.397497    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" event={"ID":"56e7c07e-0328-4d43-84e6-3017ac123d75","Type":"ContainerDied","Data":"43dec387a40ef7ab2e3ce14b182d05e669d88f4bd7def60d492e91173fbed595"}
Jul 31 23:55:36.456600 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:36.456562    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" event={"ID":"56e7c07e-0328-4d43-84e6-3017ac123d75","Type":"ContainerStarted","Data":"0ceeb5a25b89dca0c408e881fc7cf3b7d94f562c30bcf7a8d1d839d6842f4547"}
Jul 31 23:55:39.475652 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:39.475605    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" event={"ID":"56e7c07e-0328-4d43-84e6-3017ac123d75","Type":"ContainerStarted","Data":"d2a4d76757e73cace4664fec9785aa018b51aba2b63c29ce890c0cdabfa88d1e"}
Jul 31 23:55:41.485707 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:41.485669    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" event={"ID":"56e7c07e-0328-4d43-84e6-3017ac123d75","Type":"ContainerStarted","Data":"c3716e9fd23782d43a36847b57c2c057334d425a6cd01bcf871ce4ce9471122b"}
Jul 31 23:55:41.486245 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:41.485887    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:41.486245 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:41.486040    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:41.487418 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:41.487375    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 31 23:55:41.508427 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:41.508367    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podStartSLOduration=2.516802642 podStartE2EDuration="26.508350047s" totalImagesPullingTime="20.053065303s" totalInitContainerRuntime="3.938482102s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:55:15 +0000 UTC" imagePullSessionsCount=4 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:55:41.506253212 +0000 UTC m=+564.685709956" watchObservedRunningTime="2026-07-31 23:55:41.508350047 +0000 UTC m=+564.687806791"
Jul 31 23:55:42.490102 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:42.490054    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 31 23:55:42.490102 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:42.490090    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:42.491415 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:42.491384    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:55:43.493632 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:43.493591    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 31 23:55:43.494098 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:43.493930    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:55:43.497388 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:43.497369    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:55:44.496672 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:44.496630    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 31 23:55:44.497163 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:44.496948    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:55:54.497086 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:54.497037    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 31 23:55:54.497639 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:55:54.497468    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:56:04.497169 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:56:04.497104    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 31 23:56:04.497687 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:56:04.497541    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:56:14.497010 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:56:14.496936    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 31 23:56:14.497534 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:56:14.497377    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:56:24.496903 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:56:24.496849    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 31 23:56:24.497353 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:56:24.497328    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:56:34.497363 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:56:34.497299    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 31 23:56:34.499784 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:56:34.497702    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:56:44.497617 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:56:44.497516    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 31 23:56:44.498215 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:56:44.498011    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:56:54.497788 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:56:54.497754    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:56:54.498317 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:56:54.498296    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:57:00.340623 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.340584    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"]
Jul 31 23:57:00.344517 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.344490    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.347126 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.347098    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-raw-4da04-predictor-serving-cert\""
Jul 31 23:57:00.347126 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.347125    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-raw-4da04-kube-rbac-proxy-sar-config\""
Jul 31 23:57:00.354515 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.354409    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"]
Jul 31 23:57:00.377442 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.377399    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"]
Jul 31 23:57:00.377709 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.377676    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container" containerID="cri-o://0ceeb5a25b89dca0c408e881fc7cf3b7d94f562c30bcf7a8d1d839d6842f4547" gracePeriod=30
Jul 31 23:57:00.377709 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.377684    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kube-rbac-proxy" containerID="cri-o://d2a4d76757e73cace4664fec9785aa018b51aba2b63c29ce890c0cdabfa88d1e" gracePeriod=30
Jul 31 23:57:00.377897 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.377683    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="agent" containerID="cri-o://c3716e9fd23782d43a36847b57c2c057334d425a6cd01bcf871ce4ce9471122b" gracePeriod=30
Jul 31 23:57:00.435587 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.435552    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kt5q\" (UniqueName: \"kubernetes.io/projected/565239ab-1882-4c00-906f-675fb25ed64d-kube-api-access-4kt5q\") pod \"isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.435768 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.435602    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/565239ab-1882-4c00-906f-675fb25ed64d-proxy-tls\") pod \"isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.435768 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.435627    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/565239ab-1882-4c00-906f-675fb25ed64d-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.435768 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.435730    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/565239ab-1882-4c00-906f-675fb25ed64d-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.435768 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.435762    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/565239ab-1882-4c00-906f-675fb25ed64d-cabundle-cert\") pod \"isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.536489 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.536448    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/565239ab-1882-4c00-906f-675fb25ed64d-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.536489 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.536491    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/565239ab-1882-4c00-906f-675fb25ed64d-cabundle-cert\") pod \"isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.536739 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.536531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kt5q\" (UniqueName: \"kubernetes.io/projected/565239ab-1882-4c00-906f-675fb25ed64d-kube-api-access-4kt5q\") pod \"isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.536739 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.536576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/565239ab-1882-4c00-906f-675fb25ed64d-proxy-tls\") pod \"isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.536739 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.536605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/565239ab-1882-4c00-906f-675fb25ed64d-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.537076 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.537050    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/565239ab-1882-4c00-906f-675fb25ed64d-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.537269 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.537249    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/565239ab-1882-4c00-906f-675fb25ed64d-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.537323 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.537312    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/565239ab-1882-4c00-906f-675fb25ed64d-cabundle-cert\") pod \"isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.539138 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.539118    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/565239ab-1882-4c00-906f-675fb25ed64d-proxy-tls\") pod \"isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.545138 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.545117    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kt5q\" (UniqueName: \"kubernetes.io/projected/565239ab-1882-4c00-906f-675fb25ed64d-kube-api-access-4kt5q\") pod \"isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.656922 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.656869    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:00.783908 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.783875    2581 generic.go:356] "Generic (PLEG): container finished" podID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerID="d2a4d76757e73cace4664fec9785aa018b51aba2b63c29ce890c0cdabfa88d1e" exitCode=2
Jul 31 23:57:00.784087 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.783945    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" event={"ID":"56e7c07e-0328-4d43-84e6-3017ac123d75","Type":"ContainerDied","Data":"d2a4d76757e73cace4664fec9785aa018b51aba2b63c29ce890c0cdabfa88d1e"}
Jul 31 23:57:00.794476 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:00.794448    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"]
Jul 31 23:57:00.796927 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:57:00.796890    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod565239ab_1882_4c00_906f_675fb25ed64d.slice/crio-2282225068b2c08418d7c364d8addee85066f5e2630f20856faeb770f8f42845 WatchSource:0}: Error finding container 2282225068b2c08418d7c364d8addee85066f5e2630f20856faeb770f8f42845: Status 404 returned error can't find the container with id 2282225068b2c08418d7c364d8addee85066f5e2630f20856faeb770f8f42845
Jul 31 23:57:01.789624 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:01.789529    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" event={"ID":"565239ab-1882-4c00-906f-675fb25ed64d","Type":"ContainerStarted","Data":"18ece509443837a37c414c4a0c42c99074fa3f7fa49316d8f19dbf8d18b11a5e"}
Jul 31 23:57:01.790012 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:01.789645    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" event={"ID":"565239ab-1882-4c00-906f-675fb25ed64d","Type":"ContainerStarted","Data":"2282225068b2c08418d7c364d8addee85066f5e2630f20856faeb770f8f42845"}
Jul 31 23:57:03.494230 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:03.494175    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.21:8643/healthz\": dial tcp 10.133.0.21:8643: connect: connection refused"
Jul 31 23:57:04.497253 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:04.497196    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:57:04.498548 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:04.498513    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 31 23:57:04.801696 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:04.801604    2581 generic.go:356] "Generic (PLEG): container finished" podID="565239ab-1882-4c00-906f-675fb25ed64d" containerID="18ece509443837a37c414c4a0c42c99074fa3f7fa49316d8f19dbf8d18b11a5e" exitCode=0
Jul 31 23:57:04.801696 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:04.801681    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" event={"ID":"565239ab-1882-4c00-906f-675fb25ed64d","Type":"ContainerDied","Data":"18ece509443837a37c414c4a0c42c99074fa3f7fa49316d8f19dbf8d18b11a5e"}
Jul 31 23:57:05.807734 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:05.807639    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" event={"ID":"565239ab-1882-4c00-906f-675fb25ed64d","Type":"ContainerStarted","Data":"1c4f972ea61f9422efbb1f83a7c498c78584d281bc3859cc0ed96103739bf28b"}
Jul 31 23:57:05.807734 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:05.807725    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" event={"ID":"565239ab-1882-4c00-906f-675fb25ed64d","Type":"ContainerStarted","Data":"61006854b8b0a8a33db27e08bd174957f416f66893423041a021fcdd0f27c27e"}
Jul 31 23:57:05.808347 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:05.808048    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:05.808347 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:05.808190    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:05.809719 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:05.809690    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 31 23:57:05.810207 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:05.810188    2581 generic.go:356] "Generic (PLEG): container finished" podID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerID="0ceeb5a25b89dca0c408e881fc7cf3b7d94f562c30bcf7a8d1d839d6842f4547" exitCode=0
Jul 31 23:57:05.810263 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:05.810253    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" event={"ID":"56e7c07e-0328-4d43-84e6-3017ac123d75","Type":"ContainerDied","Data":"0ceeb5a25b89dca0c408e881fc7cf3b7d94f562c30bcf7a8d1d839d6842f4547"}
Jul 31 23:57:05.827524 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:05.827476    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" podStartSLOduration=1.9414731280000002 podStartE2EDuration="5.827461429s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.885988301s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:57:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:57:05.825213821 +0000 UTC m=+649.004670565" watchObservedRunningTime="2026-07-31 23:57:05.827461429 +0000 UTC m=+649.006918173"
Jul 31 23:57:06.814759 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:06.814723    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 31 23:57:08.493964 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:08.493921    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.21:8643/healthz\": dial tcp 10.133.0.21:8643: connect: connection refused"
Jul 31 23:57:11.819523 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:11.819492    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:57:11.820113 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:11.820085    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 31 23:57:13.493763 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:13.493714    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.21:8643/healthz\": dial tcp 10.133.0.21:8643: connect: connection refused"
Jul 31 23:57:13.494220 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:13.493854    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:57:14.497604 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:14.497554    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:57:14.498877 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:14.498850    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 31 23:57:18.494438 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:18.494394    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.21:8643/healthz\": dial tcp 10.133.0.21:8643: connect: connection refused"
Jul 31 23:57:21.820848 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:21.820804    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 31 23:57:23.494708 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:23.494663    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.21:8643/healthz\": dial tcp 10.133.0.21:8643: connect: connection refused"
Jul 31 23:57:24.496972 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:24.496906    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:57:24.497392 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:24.497088    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:57:24.498372 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:24.498348    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 31 23:57:24.498493 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:24.498445    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:57:28.494582 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:28.494537    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.21:8643/healthz\": dial tcp 10.133.0.21:8643: connect: connection refused"
Jul 31 23:57:30.534944 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.534910    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:57:30.551593 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.551565    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56e7c07e-0328-4d43-84e6-3017ac123d75-cabundle-cert\") pod \"56e7c07e-0328-4d43-84e6-3017ac123d75\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") "
Jul 31 23:57:30.551593 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.551606    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56e7c07e-0328-4d43-84e6-3017ac123d75-proxy-tls\") pod \"56e7c07e-0328-4d43-84e6-3017ac123d75\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") "
Jul 31 23:57:30.551784 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.551623    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56e7c07e-0328-4d43-84e6-3017ac123d75-kube-rbac-proxy-sar-config\") pod \"56e7c07e-0328-4d43-84e6-3017ac123d75\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") "
Jul 31 23:57:30.551841 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.551774    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56e7c07e-0328-4d43-84e6-3017ac123d75-kserve-provision-location\") pod \"56e7c07e-0328-4d43-84e6-3017ac123d75\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") "
Jul 31 23:57:30.552033 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.552009    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56e7c07e-0328-4d43-84e6-3017ac123d75-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "56e7c07e-0328-4d43-84e6-3017ac123d75" (UID: "56e7c07e-0328-4d43-84e6-3017ac123d75"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:57:30.552033 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.552020    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56e7c07e-0328-4d43-84e6-3017ac123d75-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "56e7c07e-0328-4d43-84e6-3017ac123d75" (UID: "56e7c07e-0328-4d43-84e6-3017ac123d75"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:57:30.552161 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.552059    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56e7c07e-0328-4d43-84e6-3017ac123d75-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "56e7c07e-0328-4d43-84e6-3017ac123d75" (UID: "56e7c07e-0328-4d43-84e6-3017ac123d75"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 23:57:30.553878 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.553856    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56e7c07e-0328-4d43-84e6-3017ac123d75-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "56e7c07e-0328-4d43-84e6-3017ac123d75" (UID: "56e7c07e-0328-4d43-84e6-3017ac123d75"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:57:30.652386 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.652288    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lm62z\" (UniqueName: \"kubernetes.io/projected/56e7c07e-0328-4d43-84e6-3017ac123d75-kube-api-access-lm62z\") pod \"56e7c07e-0328-4d43-84e6-3017ac123d75\" (UID: \"56e7c07e-0328-4d43-84e6-3017ac123d75\") "
Jul 31 23:57:30.652534 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.652427    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56e7c07e-0328-4d43-84e6-3017ac123d75-kserve-provision-location\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:57:30.652534 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.652440    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56e7c07e-0328-4d43-84e6-3017ac123d75-cabundle-cert\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:57:30.652534 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.652449    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56e7c07e-0328-4d43-84e6-3017ac123d75-proxy-tls\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:57:30.652534 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.652458    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56e7c07e-0328-4d43-84e6-3017ac123d75-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:57:30.654297 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.654271    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56e7c07e-0328-4d43-84e6-3017ac123d75-kube-api-access-lm62z" (OuterVolumeSpecName: "kube-api-access-lm62z") pod "56e7c07e-0328-4d43-84e6-3017ac123d75" (UID: "56e7c07e-0328-4d43-84e6-3017ac123d75"). InnerVolumeSpecName "kube-api-access-lm62z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:57:30.753575 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.753537    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lm62z\" (UniqueName: \"kubernetes.io/projected/56e7c07e-0328-4d43-84e6-3017ac123d75-kube-api-access-lm62z\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:57:30.902904 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.902822    2581 generic.go:356] "Generic (PLEG): container finished" podID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerID="c3716e9fd23782d43a36847b57c2c057334d425a6cd01bcf871ce4ce9471122b" exitCode=0
Jul 31 23:57:30.902904 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.902873    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" event={"ID":"56e7c07e-0328-4d43-84e6-3017ac123d75","Type":"ContainerDied","Data":"c3716e9fd23782d43a36847b57c2c057334d425a6cd01bcf871ce4ce9471122b"}
Jul 31 23:57:30.902904 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.902884    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"
Jul 31 23:57:30.903143 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.902928    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv" event={"ID":"56e7c07e-0328-4d43-84e6-3017ac123d75","Type":"ContainerDied","Data":"16b66d877dc2cd60a5cdd69fac91143d892b44d07d2dc670ee817b00435dab37"}
Jul 31 23:57:30.903143 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.902977    2581 scope.go:117] "RemoveContainer" containerID="c3716e9fd23782d43a36847b57c2c057334d425a6cd01bcf871ce4ce9471122b"
Jul 31 23:57:30.911140 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.911122    2581 scope.go:117] "RemoveContainer" containerID="d2a4d76757e73cace4664fec9785aa018b51aba2b63c29ce890c0cdabfa88d1e"
Jul 31 23:57:30.918148 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.918124    2581 scope.go:117] "RemoveContainer" containerID="0ceeb5a25b89dca0c408e881fc7cf3b7d94f562c30bcf7a8d1d839d6842f4547"
Jul 31 23:57:30.924249 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.924226    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"]
Jul 31 23:57:30.925502 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.925483    2581 scope.go:117] "RemoveContainer" containerID="43dec387a40ef7ab2e3ce14b182d05e669d88f4bd7def60d492e91173fbed595"
Jul 31 23:57:30.929527 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.929504    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-raw-sklearn-batcher-76a60-predictor-79d4b46578-zlgpv"]
Jul 31 23:57:30.933240 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.933220    2581 scope.go:117] "RemoveContainer" containerID="c3716e9fd23782d43a36847b57c2c057334d425a6cd01bcf871ce4ce9471122b"
Jul 31 23:57:30.933535 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:57:30.933518    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3716e9fd23782d43a36847b57c2c057334d425a6cd01bcf871ce4ce9471122b\": container with ID starting with c3716e9fd23782d43a36847b57c2c057334d425a6cd01bcf871ce4ce9471122b not found: ID does not exist" containerID="c3716e9fd23782d43a36847b57c2c057334d425a6cd01bcf871ce4ce9471122b"
Jul 31 23:57:30.933583 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.933544    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3716e9fd23782d43a36847b57c2c057334d425a6cd01bcf871ce4ce9471122b"} err="failed to get container status \"c3716e9fd23782d43a36847b57c2c057334d425a6cd01bcf871ce4ce9471122b\": rpc error: code = NotFound desc = could not find container \"c3716e9fd23782d43a36847b57c2c057334d425a6cd01bcf871ce4ce9471122b\": container with ID starting with c3716e9fd23782d43a36847b57c2c057334d425a6cd01bcf871ce4ce9471122b not found: ID does not exist"
Jul 31 23:57:30.933583 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.933562    2581 scope.go:117] "RemoveContainer" containerID="d2a4d76757e73cace4664fec9785aa018b51aba2b63c29ce890c0cdabfa88d1e"
Jul 31 23:57:30.933821 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:57:30.933808    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2a4d76757e73cace4664fec9785aa018b51aba2b63c29ce890c0cdabfa88d1e\": container with ID starting with d2a4d76757e73cace4664fec9785aa018b51aba2b63c29ce890c0cdabfa88d1e not found: ID does not exist" containerID="d2a4d76757e73cace4664fec9785aa018b51aba2b63c29ce890c0cdabfa88d1e"
Jul 31 23:57:30.933866 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.933825    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2a4d76757e73cace4664fec9785aa018b51aba2b63c29ce890c0cdabfa88d1e"} err="failed to get container status \"d2a4d76757e73cace4664fec9785aa018b51aba2b63c29ce890c0cdabfa88d1e\": rpc error: code = NotFound desc = could not find container \"d2a4d76757e73cace4664fec9785aa018b51aba2b63c29ce890c0cdabfa88d1e\": container with ID starting with d2a4d76757e73cace4664fec9785aa018b51aba2b63c29ce890c0cdabfa88d1e not found: ID does not exist"
Jul 31 23:57:30.933866 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.933839    2581 scope.go:117] "RemoveContainer" containerID="0ceeb5a25b89dca0c408e881fc7cf3b7d94f562c30bcf7a8d1d839d6842f4547"
Jul 31 23:57:30.934063 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:57:30.934049    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ceeb5a25b89dca0c408e881fc7cf3b7d94f562c30bcf7a8d1d839d6842f4547\": container with ID starting with 0ceeb5a25b89dca0c408e881fc7cf3b7d94f562c30bcf7a8d1d839d6842f4547 not found: ID does not exist" containerID="0ceeb5a25b89dca0c408e881fc7cf3b7d94f562c30bcf7a8d1d839d6842f4547"
Jul 31 23:57:30.934114 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.934067    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ceeb5a25b89dca0c408e881fc7cf3b7d94f562c30bcf7a8d1d839d6842f4547"} err="failed to get container status \"0ceeb5a25b89dca0c408e881fc7cf3b7d94f562c30bcf7a8d1d839d6842f4547\": rpc error: code = NotFound desc = could not find container \"0ceeb5a25b89dca0c408e881fc7cf3b7d94f562c30bcf7a8d1d839d6842f4547\": container with ID starting with 0ceeb5a25b89dca0c408e881fc7cf3b7d94f562c30bcf7a8d1d839d6842f4547 not found: ID does not exist"
Jul 31 23:57:30.934114 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.934082    2581 scope.go:117] "RemoveContainer" containerID="43dec387a40ef7ab2e3ce14b182d05e669d88f4bd7def60d492e91173fbed595"
Jul 31 23:57:30.934333 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:57:30.934318    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43dec387a40ef7ab2e3ce14b182d05e669d88f4bd7def60d492e91173fbed595\": container with ID starting with 43dec387a40ef7ab2e3ce14b182d05e669d88f4bd7def60d492e91173fbed595 not found: ID does not exist" containerID="43dec387a40ef7ab2e3ce14b182d05e669d88f4bd7def60d492e91173fbed595"
Jul 31 23:57:30.934385 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:30.934337    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43dec387a40ef7ab2e3ce14b182d05e669d88f4bd7def60d492e91173fbed595"} err="failed to get container status \"43dec387a40ef7ab2e3ce14b182d05e669d88f4bd7def60d492e91173fbed595\": rpc error: code = NotFound desc = could not find container \"43dec387a40ef7ab2e3ce14b182d05e669d88f4bd7def60d492e91173fbed595\": container with ID starting with 43dec387a40ef7ab2e3ce14b182d05e669d88f4bd7def60d492e91173fbed595 not found: ID does not exist"
Jul 31 23:57:31.386795 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:31.386756    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" path="/var/lib/kubelet/pods/56e7c07e-0328-4d43-84e6-3017ac123d75/volumes"
Jul 31 23:57:31.820237 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:31.820142    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 31 23:57:41.820260 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:41.820214    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 31 23:57:51.820694 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:57:51.820649    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 31 23:58:01.820547 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:01.820452    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 31 23:58:11.821017 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:11.820915    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 31 23:58:21.820499 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:21.820465    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:58:40.364215 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.364179    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"]
Jul 31 23:58:40.364629 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.364478    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container"
Jul 31 23:58:40.364629 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.364490    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container"
Jul 31 23:58:40.364629 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.364498    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kube-rbac-proxy"
Jul 31 23:58:40.364629 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.364503    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kube-rbac-proxy"
Jul 31 23:58:40.364629 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.364510    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="agent"
Jul 31 23:58:40.364629 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.364515    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="agent"
Jul 31 23:58:40.364629 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.364531    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="storage-initializer"
Jul 31 23:58:40.364629 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.364540    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="storage-initializer"
Jul 31 23:58:40.364629 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.364582    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kserve-container"
Jul 31 23:58:40.364629 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.364589    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="kube-rbac-proxy"
Jul 31 23:58:40.364629 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.364594    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="56e7c07e-0328-4d43-84e6-3017ac123d75" containerName="agent"
Jul 31 23:58:40.368578 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.368559    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"
Jul 31 23:58:40.370897 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.370869    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-4da04-serving-cert\""
Jul 31 23:58:40.371010 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.370923    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-4da04-kube-rbac-proxy-sar-config\""
Jul 31 23:58:40.382895 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.378789    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"]
Jul 31 23:58:40.403444 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.403403    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e16ae90-8834-4076-8587-5972a34028e0-openshift-service-ca-bundle\") pod \"model-chainer-raw-4da04-6986b69d76-nwhcd\" (UID: \"6e16ae90-8834-4076-8587-5972a34028e0\") " pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"
Jul 31 23:58:40.403624 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.403483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6e16ae90-8834-4076-8587-5972a34028e0-proxy-tls\") pod \"model-chainer-raw-4da04-6986b69d76-nwhcd\" (UID: \"6e16ae90-8834-4076-8587-5972a34028e0\") " pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"
Jul 31 23:58:40.504156 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.504117    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6e16ae90-8834-4076-8587-5972a34028e0-proxy-tls\") pod \"model-chainer-raw-4da04-6986b69d76-nwhcd\" (UID: \"6e16ae90-8834-4076-8587-5972a34028e0\") " pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"
Jul 31 23:58:40.504356 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.504169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e16ae90-8834-4076-8587-5972a34028e0-openshift-service-ca-bundle\") pod \"model-chainer-raw-4da04-6986b69d76-nwhcd\" (UID: \"6e16ae90-8834-4076-8587-5972a34028e0\") " pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"
Jul 31 23:58:40.504766 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.504747    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e16ae90-8834-4076-8587-5972a34028e0-openshift-service-ca-bundle\") pod \"model-chainer-raw-4da04-6986b69d76-nwhcd\" (UID: \"6e16ae90-8834-4076-8587-5972a34028e0\") " pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"
Jul 31 23:58:40.506656 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.506629    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6e16ae90-8834-4076-8587-5972a34028e0-proxy-tls\") pod \"model-chainer-raw-4da04-6986b69d76-nwhcd\" (UID: \"6e16ae90-8834-4076-8587-5972a34028e0\") " pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"
Jul 31 23:58:40.680013 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.679944    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"
Jul 31 23:58:40.811599 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.811575    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"]
Jul 31 23:58:40.813839 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:58:40.813807    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e16ae90_8834_4076_8587_5972a34028e0.slice/crio-eb3d0a1451206041dbdfbb69ddf9b2c95825ed7089329f952fa0f9d0ec7db123 WatchSource:0}: Error finding container eb3d0a1451206041dbdfbb69ddf9b2c95825ed7089329f952fa0f9d0ec7db123: Status 404 returned error can't find the container with id eb3d0a1451206041dbdfbb69ddf9b2c95825ed7089329f952fa0f9d0ec7db123
Jul 31 23:58:40.815774 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:40.815756    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 23:58:41.157261 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:41.157221    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd" event={"ID":"6e16ae90-8834-4076-8587-5972a34028e0","Type":"ContainerStarted","Data":"eb3d0a1451206041dbdfbb69ddf9b2c95825ed7089329f952fa0f9d0ec7db123"}
Jul 31 23:58:43.166416 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:43.166375    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd" event={"ID":"6e16ae90-8834-4076-8587-5972a34028e0","Type":"ContainerStarted","Data":"8a72041b7a8a355fc52fd4e1f416e6f813b2bab62f1ac4f3a339a77bb79df13d"}
Jul 31 23:58:43.166828 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:43.166487    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"
Jul 31 23:58:43.182878 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:43.182831    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd" podStartSLOduration=1.288639684 podStartE2EDuration="3.18281352s" totalImagesPullingTime="1.894173836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:58:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:58:43.181982736 +0000 UTC m=+746.361439483" watchObservedRunningTime="2026-07-31 23:58:43.18281352 +0000 UTC m=+746.362270263"
Jul 31 23:58:49.176120 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:49.176088    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"
Jul 31 23:58:50.412243 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.412209    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"]
Jul 31 23:58:50.412622 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.412417    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd" podUID="6e16ae90-8834-4076-8587-5972a34028e0" containerName="model-chainer-raw-4da04" containerID="cri-o://8a72041b7a8a355fc52fd4e1f416e6f813b2bab62f1ac4f3a339a77bb79df13d" gracePeriod=30
Jul 31 23:58:50.534428 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.534395    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"]
Jul 31 23:58:50.534681 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.534636    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kserve-container" containerID="cri-o://61006854b8b0a8a33db27e08bd174957f416f66893423041a021fcdd0f27c27e" gracePeriod=30
Jul 31 23:58:50.534760 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.534705    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kube-rbac-proxy" containerID="cri-o://1c4f972ea61f9422efbb1f83a7c498c78584d281bc3859cc0ed96103739bf28b" gracePeriod=30
Jul 31 23:58:50.671015 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.670916    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"]
Jul 31 23:58:50.674577 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.674373    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:50.678053 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.678016    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-raw-hpa-64006-predictor-serving-cert\""
Jul 31 23:58:50.678053 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.678030    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-raw-hpa-64006-kube-rbac-proxy-sar-config\""
Jul 31 23:58:50.684093 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.684013    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"]
Jul 31 23:58:50.782736 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.782703    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7aeed7fb-652f-4ea1-adf7-442ba749345d-proxy-tls\") pod \"isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:50.782736 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.782745    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcxbl\" (UniqueName: \"kubernetes.io/projected/7aeed7fb-652f-4ea1-adf7-442ba749345d-kube-api-access-gcxbl\") pod \"isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:50.782984 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.782790    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7aeed7fb-652f-4ea1-adf7-442ba749345d-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:50.782984 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.782827    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7aeed7fb-652f-4ea1-adf7-442ba749345d-cabundle-cert\") pod \"isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:50.782984 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.782912    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7aeed7fb-652f-4ea1-adf7-442ba749345d-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:50.884082 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.884045    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7aeed7fb-652f-4ea1-adf7-442ba749345d-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:50.884082 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.884084    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7aeed7fb-652f-4ea1-adf7-442ba749345d-cabundle-cert\") pod \"isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:50.884298 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.884217    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7aeed7fb-652f-4ea1-adf7-442ba749345d-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:50.884298 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.884290    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7aeed7fb-652f-4ea1-adf7-442ba749345d-proxy-tls\") pod \"isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:50.884396 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.884327    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcxbl\" (UniqueName: \"kubernetes.io/projected/7aeed7fb-652f-4ea1-adf7-442ba749345d-kube-api-access-gcxbl\") pod \"isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:50.884551 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.884518    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7aeed7fb-652f-4ea1-adf7-442ba749345d-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:50.884845 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.884824    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7aeed7fb-652f-4ea1-adf7-442ba749345d-cabundle-cert\") pod \"isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:50.884928 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.884826    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7aeed7fb-652f-4ea1-adf7-442ba749345d-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:50.886790 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.886768    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7aeed7fb-652f-4ea1-adf7-442ba749345d-proxy-tls\") pod \"isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:50.892021 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.891997    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcxbl\" (UniqueName: \"kubernetes.io/projected/7aeed7fb-652f-4ea1-adf7-442ba749345d-kube-api-access-gcxbl\") pod \"isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:50.987038 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:50.986922    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:51.122589 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:51.122553    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"]
Jul 31 23:58:51.126229 ip-10-0-139-141 kubenswrapper[2581]: W0731 23:58:51.126188    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aeed7fb_652f_4ea1_adf7_442ba749345d.slice/crio-ae7102fb197bc82708061a5d7a91d925aba48551d15679e271b5670698191ddb WatchSource:0}: Error finding container ae7102fb197bc82708061a5d7a91d925aba48551d15679e271b5670698191ddb: Status 404 returned error can't find the container with id ae7102fb197bc82708061a5d7a91d925aba48551d15679e271b5670698191ddb
Jul 31 23:58:51.196397 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:51.196340    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" event={"ID":"7aeed7fb-652f-4ea1-adf7-442ba749345d","Type":"ContainerStarted","Data":"c186a8420ed61766cdda61a27675487d7e2f70b425d0726f547a6f6aba5a2c52"}
Jul 31 23:58:51.196583 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:51.196422    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" event={"ID":"7aeed7fb-652f-4ea1-adf7-442ba749345d","Type":"ContainerStarted","Data":"ae7102fb197bc82708061a5d7a91d925aba48551d15679e271b5670698191ddb"}
Jul 31 23:58:51.198767 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:51.198739    2581 generic.go:356] "Generic (PLEG): container finished" podID="565239ab-1882-4c00-906f-675fb25ed64d" containerID="1c4f972ea61f9422efbb1f83a7c498c78584d281bc3859cc0ed96103739bf28b" exitCode=2
Jul 31 23:58:51.198888 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:51.198794    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" event={"ID":"565239ab-1882-4c00-906f-675fb25ed64d","Type":"ContainerDied","Data":"1c4f972ea61f9422efbb1f83a7c498c78584d281bc3859cc0ed96103739bf28b"}
Jul 31 23:58:51.815694 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:51.815650    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.22:8643/healthz\": dial tcp 10.133.0.22:8643: connect: connection refused"
Jul 31 23:58:51.820661 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:51.820624    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 31 23:58:54.174570 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:54.174516    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd" podUID="6e16ae90-8834-4076-8587-5972a34028e0" containerName="model-chainer-raw-4da04" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:58:55.215756 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.215723    2581 generic.go:356] "Generic (PLEG): container finished" podID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerID="c186a8420ed61766cdda61a27675487d7e2f70b425d0726f547a6f6aba5a2c52" exitCode=0
Jul 31 23:58:55.216158 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.215795    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" event={"ID":"7aeed7fb-652f-4ea1-adf7-442ba749345d","Type":"ContainerDied","Data":"c186a8420ed61766cdda61a27675487d7e2f70b425d0726f547a6f6aba5a2c52"}
Jul 31 23:58:55.485145 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.485115    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:58:55.518643 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.518609    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kt5q\" (UniqueName: \"kubernetes.io/projected/565239ab-1882-4c00-906f-675fb25ed64d-kube-api-access-4kt5q\") pod \"565239ab-1882-4c00-906f-675fb25ed64d\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") "
Jul 31 23:58:55.518812 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.518666    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/565239ab-1882-4c00-906f-675fb25ed64d-cabundle-cert\") pod \"565239ab-1882-4c00-906f-675fb25ed64d\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") "
Jul 31 23:58:55.518812 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.518696    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/565239ab-1882-4c00-906f-675fb25ed64d-kube-rbac-proxy-sar-config\") pod \"565239ab-1882-4c00-906f-675fb25ed64d\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") "
Jul 31 23:58:55.518812 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.518734    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/565239ab-1882-4c00-906f-675fb25ed64d-proxy-tls\") pod \"565239ab-1882-4c00-906f-675fb25ed64d\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") "
Jul 31 23:58:55.518812 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.518777    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/565239ab-1882-4c00-906f-675fb25ed64d-kserve-provision-location\") pod \"565239ab-1882-4c00-906f-675fb25ed64d\" (UID: \"565239ab-1882-4c00-906f-675fb25ed64d\") "
Jul 31 23:58:55.519064 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.519034    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/565239ab-1882-4c00-906f-675fb25ed64d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "565239ab-1882-4c00-906f-675fb25ed64d" (UID: "565239ab-1882-4c00-906f-675fb25ed64d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:58:55.519193 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.519165    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/565239ab-1882-4c00-906f-675fb25ed64d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:58:55.519259 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.519166    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/565239ab-1882-4c00-906f-675fb25ed64d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "565239ab-1882-4c00-906f-675fb25ed64d" (UID: "565239ab-1882-4c00-906f-675fb25ed64d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 23:58:55.519259 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.519234    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/565239ab-1882-4c00-906f-675fb25ed64d-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "565239ab-1882-4c00-906f-675fb25ed64d" (UID: "565239ab-1882-4c00-906f-675fb25ed64d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:58:55.520776 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.520754    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/565239ab-1882-4c00-906f-675fb25ed64d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "565239ab-1882-4c00-906f-675fb25ed64d" (UID: "565239ab-1882-4c00-906f-675fb25ed64d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:58:55.520776 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.520763    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/565239ab-1882-4c00-906f-675fb25ed64d-kube-api-access-4kt5q" (OuterVolumeSpecName: "kube-api-access-4kt5q") pod "565239ab-1882-4c00-906f-675fb25ed64d" (UID: "565239ab-1882-4c00-906f-675fb25ed64d"). InnerVolumeSpecName "kube-api-access-4kt5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:58:55.620101 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.620057    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kt5q\" (UniqueName: \"kubernetes.io/projected/565239ab-1882-4c00-906f-675fb25ed64d-kube-api-access-4kt5q\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:58:55.620101 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.620089    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/565239ab-1882-4c00-906f-675fb25ed64d-cabundle-cert\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:58:55.620101 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.620098    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/565239ab-1882-4c00-906f-675fb25ed64d-proxy-tls\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:58:55.620101 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:55.620107    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/565239ab-1882-4c00-906f-675fb25ed64d-kserve-provision-location\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:58:56.221304 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.221265    2581 generic.go:356] "Generic (PLEG): container finished" podID="565239ab-1882-4c00-906f-675fb25ed64d" containerID="61006854b8b0a8a33db27e08bd174957f416f66893423041a021fcdd0f27c27e" exitCode=0
Jul 31 23:58:56.221724 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.221315    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"
Jul 31 23:58:56.221724 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.221345    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" event={"ID":"565239ab-1882-4c00-906f-675fb25ed64d","Type":"ContainerDied","Data":"61006854b8b0a8a33db27e08bd174957f416f66893423041a021fcdd0f27c27e"}
Jul 31 23:58:56.221724 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.221412    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7" event={"ID":"565239ab-1882-4c00-906f-675fb25ed64d","Type":"ContainerDied","Data":"2282225068b2c08418d7c364d8addee85066f5e2630f20856faeb770f8f42845"}
Jul 31 23:58:56.221724 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.221444    2581 scope.go:117] "RemoveContainer" containerID="1c4f972ea61f9422efbb1f83a7c498c78584d281bc3859cc0ed96103739bf28b"
Jul 31 23:58:56.223792 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.223760    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" event={"ID":"7aeed7fb-652f-4ea1-adf7-442ba749345d","Type":"ContainerStarted","Data":"ebb96a2f15a8c1b14f8c8eed0dd63d428d792a2b53678cfd63db0d3ee249e405"}
Jul 31 23:58:56.223885 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.223842    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" event={"ID":"7aeed7fb-652f-4ea1-adf7-442ba749345d","Type":"ContainerStarted","Data":"35b48e4266a8bed8ada8ada8ada425947d9012ba465bd062e713d340842ee03a"}
Jul 31 23:58:56.224134 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.224114    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:56.230222 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.230188    2581 scope.go:117] "RemoveContainer" containerID="61006854b8b0a8a33db27e08bd174957f416f66893423041a021fcdd0f27c27e"
Jul 31 23:58:56.238620 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.238596    2581 scope.go:117] "RemoveContainer" containerID="18ece509443837a37c414c4a0c42c99074fa3f7fa49316d8f19dbf8d18b11a5e"
Jul 31 23:58:56.246038 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.245976    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" podStartSLOduration=2.6497950120000002 podStartE2EDuration="6.245944036s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.596149024s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:58:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:58:56.245369929 +0000 UTC m=+759.424826673" watchObservedRunningTime="2026-07-31 23:58:56.245944036 +0000 UTC m=+759.425400780"
Jul 31 23:58:56.246038 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.246032    2581 scope.go:117] "RemoveContainer" containerID="1c4f972ea61f9422efbb1f83a7c498c78584d281bc3859cc0ed96103739bf28b"
Jul 31 23:58:56.246388 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:58:56.246367    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c4f972ea61f9422efbb1f83a7c498c78584d281bc3859cc0ed96103739bf28b\": container with ID starting with 1c4f972ea61f9422efbb1f83a7c498c78584d281bc3859cc0ed96103739bf28b not found: ID does not exist" containerID="1c4f972ea61f9422efbb1f83a7c498c78584d281bc3859cc0ed96103739bf28b"
Jul 31 23:58:56.246492 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.246394    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c4f972ea61f9422efbb1f83a7c498c78584d281bc3859cc0ed96103739bf28b"} err="failed to get container status \"1c4f972ea61f9422efbb1f83a7c498c78584d281bc3859cc0ed96103739bf28b\": rpc error: code = NotFound desc = could not find container \"1c4f972ea61f9422efbb1f83a7c498c78584d281bc3859cc0ed96103739bf28b\": container with ID starting with 1c4f972ea61f9422efbb1f83a7c498c78584d281bc3859cc0ed96103739bf28b not found: ID does not exist"
Jul 31 23:58:56.246492 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.246418    2581 scope.go:117] "RemoveContainer" containerID="61006854b8b0a8a33db27e08bd174957f416f66893423041a021fcdd0f27c27e"
Jul 31 23:58:56.246667 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:58:56.246643    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61006854b8b0a8a33db27e08bd174957f416f66893423041a021fcdd0f27c27e\": container with ID starting with 61006854b8b0a8a33db27e08bd174957f416f66893423041a021fcdd0f27c27e not found: ID does not exist" containerID="61006854b8b0a8a33db27e08bd174957f416f66893423041a021fcdd0f27c27e"
Jul 31 23:58:56.246726 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.246676    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61006854b8b0a8a33db27e08bd174957f416f66893423041a021fcdd0f27c27e"} err="failed to get container status \"61006854b8b0a8a33db27e08bd174957f416f66893423041a021fcdd0f27c27e\": rpc error: code = NotFound desc = could not find container \"61006854b8b0a8a33db27e08bd174957f416f66893423041a021fcdd0f27c27e\": container with ID starting with 61006854b8b0a8a33db27e08bd174957f416f66893423041a021fcdd0f27c27e not found: ID does not exist"
Jul 31 23:58:56.246726 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.246693    2581 scope.go:117] "RemoveContainer" containerID="18ece509443837a37c414c4a0c42c99074fa3f7fa49316d8f19dbf8d18b11a5e"
Jul 31 23:58:56.246937 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:58:56.246921    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18ece509443837a37c414c4a0c42c99074fa3f7fa49316d8f19dbf8d18b11a5e\": container with ID starting with 18ece509443837a37c414c4a0c42c99074fa3f7fa49316d8f19dbf8d18b11a5e not found: ID does not exist" containerID="18ece509443837a37c414c4a0c42c99074fa3f7fa49316d8f19dbf8d18b11a5e"
Jul 31 23:58:56.247076 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.246940    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18ece509443837a37c414c4a0c42c99074fa3f7fa49316d8f19dbf8d18b11a5e"} err="failed to get container status \"18ece509443837a37c414c4a0c42c99074fa3f7fa49316d8f19dbf8d18b11a5e\": rpc error: code = NotFound desc = could not find container \"18ece509443837a37c414c4a0c42c99074fa3f7fa49316d8f19dbf8d18b11a5e\": container with ID starting with 18ece509443837a37c414c4a0c42c99074fa3f7fa49316d8f19dbf8d18b11a5e not found: ID does not exist"
Jul 31 23:58:56.258239 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.258215    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"]
Jul 31 23:58:56.262874 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:56.262854    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-4da04-predictor-66fb59fdb-5vmk7"]
Jul 31 23:58:57.229416 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:57.229383    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:58:57.230821 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:57.230789    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 31 23:58:57.387357 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:57.387324    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="565239ab-1882-4c00-906f-675fb25ed64d" path="/var/lib/kubelet/pods/565239ab-1882-4c00-906f-675fb25ed64d/volumes"
Jul 31 23:58:58.233912 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:58.233865    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 31 23:58:59.174773 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:58:59.174726    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd" podUID="6e16ae90-8834-4076-8587-5972a34028e0" containerName="model-chainer-raw-4da04" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:59:03.239283 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:03.239256    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Jul 31 23:59:03.239823 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:03.239790    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 31 23:59:04.174507 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:04.174467    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd" podUID="6e16ae90-8834-4076-8587-5972a34028e0" containerName="model-chainer-raw-4da04" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:59:04.174686 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:04.174583    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"
Jul 31 23:59:09.175044 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:09.174998    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd" podUID="6e16ae90-8834-4076-8587-5972a34028e0" containerName="model-chainer-raw-4da04" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:59:13.239941 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:13.239896    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 31 23:59:14.174748 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:14.174709    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd" podUID="6e16ae90-8834-4076-8587-5972a34028e0" containerName="model-chainer-raw-4da04" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:59:19.174254 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:19.174213    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd" podUID="6e16ae90-8834-4076-8587-5972a34028e0" containerName="model-chainer-raw-4da04" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:59:20.562042 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:20.562018    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"
Jul 31 23:59:20.623100 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:20.623057    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e16ae90-8834-4076-8587-5972a34028e0-openshift-service-ca-bundle\") pod \"6e16ae90-8834-4076-8587-5972a34028e0\" (UID: \"6e16ae90-8834-4076-8587-5972a34028e0\") "
Jul 31 23:59:20.623100 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:20.623105    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6e16ae90-8834-4076-8587-5972a34028e0-proxy-tls\") pod \"6e16ae90-8834-4076-8587-5972a34028e0\" (UID: \"6e16ae90-8834-4076-8587-5972a34028e0\") "
Jul 31 23:59:20.623472 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:20.623451    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e16ae90-8834-4076-8587-5972a34028e0-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "6e16ae90-8834-4076-8587-5972a34028e0" (UID: "6e16ae90-8834-4076-8587-5972a34028e0"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:59:20.625306 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:20.625283    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e16ae90-8834-4076-8587-5972a34028e0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6e16ae90-8834-4076-8587-5972a34028e0" (UID: "6e16ae90-8834-4076-8587-5972a34028e0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:59:20.724545 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:20.724463    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e16ae90-8834-4076-8587-5972a34028e0-openshift-service-ca-bundle\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:59:20.724545 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:20.724492    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6e16ae90-8834-4076-8587-5972a34028e0-proxy-tls\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Jul 31 23:59:21.319066 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:21.319030    2581 generic.go:356] "Generic (PLEG): container finished" podID="6e16ae90-8834-4076-8587-5972a34028e0" containerID="8a72041b7a8a355fc52fd4e1f416e6f813b2bab62f1ac4f3a339a77bb79df13d" exitCode=0
Jul 31 23:59:21.319239 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:21.319078    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"
Jul 31 23:59:21.319239 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:21.319119    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd" event={"ID":"6e16ae90-8834-4076-8587-5972a34028e0","Type":"ContainerDied","Data":"8a72041b7a8a355fc52fd4e1f416e6f813b2bab62f1ac4f3a339a77bb79df13d"}
Jul 31 23:59:21.319239 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:21.319181    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd" event={"ID":"6e16ae90-8834-4076-8587-5972a34028e0","Type":"ContainerDied","Data":"eb3d0a1451206041dbdfbb69ddf9b2c95825ed7089329f952fa0f9d0ec7db123"}
Jul 31 23:59:21.319239 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:21.319209    2581 scope.go:117] "RemoveContainer" containerID="8a72041b7a8a355fc52fd4e1f416e6f813b2bab62f1ac4f3a339a77bb79df13d"
Jul 31 23:59:21.327621 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:21.327595    2581 scope.go:117] "RemoveContainer" containerID="8a72041b7a8a355fc52fd4e1f416e6f813b2bab62f1ac4f3a339a77bb79df13d"
Jul 31 23:59:21.327963 ip-10-0-139-141 kubenswrapper[2581]: E0731 23:59:21.327931    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a72041b7a8a355fc52fd4e1f416e6f813b2bab62f1ac4f3a339a77bb79df13d\": container with ID starting with 8a72041b7a8a355fc52fd4e1f416e6f813b2bab62f1ac4f3a339a77bb79df13d not found: ID does not exist" containerID="8a72041b7a8a355fc52fd4e1f416e6f813b2bab62f1ac4f3a339a77bb79df13d"
Jul 31 23:59:21.328073 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:21.327982    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a72041b7a8a355fc52fd4e1f416e6f813b2bab62f1ac4f3a339a77bb79df13d"} err="failed to get container status \"8a72041b7a8a355fc52fd4e1f416e6f813b2bab62f1ac4f3a339a77bb79df13d\": rpc error: code = NotFound desc = could not find container \"8a72041b7a8a355fc52fd4e1f416e6f813b2bab62f1ac4f3a339a77bb79df13d\": container with ID starting with 8a72041b7a8a355fc52fd4e1f416e6f813b2bab62f1ac4f3a339a77bb79df13d not found: ID does not exist"
Jul 31 23:59:21.339608 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:21.339575    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"]
Jul 31 23:59:21.342970 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:21.342923    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-4da04-6986b69d76-nwhcd"]
Jul 31 23:59:21.387394 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:21.387360    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e16ae90-8834-4076-8587-5972a34028e0" path="/var/lib/kubelet/pods/6e16ae90-8834-4076-8587-5972a34028e0/volumes"
Jul 31 23:59:23.240419 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:23.240373    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 31 23:59:33.240341 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:33.240280    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 31 23:59:43.240258 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:43.240165    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 31 23:59:53.239807 ip-10-0-139-141 kubenswrapper[2581]: I0731 23:59:53.239766    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Aug 01 00:00:00.144762 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.144718    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-pruner-29759040-jszhz"]
Aug 01 00:00:00.145166 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.145002    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e16ae90-8834-4076-8587-5972a34028e0" containerName="model-chainer-raw-4da04"
Aug 01 00:00:00.145166 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.145014    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e16ae90-8834-4076-8587-5972a34028e0" containerName="model-chainer-raw-4da04"
Aug 01 00:00:00.145166 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.145022    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kube-rbac-proxy"
Aug 01 00:00:00.145166 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.145028    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kube-rbac-proxy"
Aug 01 00:00:00.145166 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.145037    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="storage-initializer"
Aug 01 00:00:00.145166 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.145043    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="storage-initializer"
Aug 01 00:00:00.145166 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.145058    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kserve-container"
Aug 01 00:00:00.145166 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.145063    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kserve-container"
Aug 01 00:00:00.145166 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.145102    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kserve-container"
Aug 01 00:00:00.145166 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.145109    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="565239ab-1882-4c00-906f-675fb25ed64d" containerName="kube-rbac-proxy"
Aug 01 00:00:00.145166 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.145116    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e16ae90-8834-4076-8587-5972a34028e0" containerName="model-chainer-raw-4da04"
Aug 01 00:00:00.147950 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.147933    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29759040-jszhz"
Aug 01 00:00:00.151044 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.151022    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"serviceca\""
Aug 01 00:00:00.151156 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.151026    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"pruner-dockercfg-pbnhn\""
Aug 01 00:00:00.157512 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.157488    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29759040-jszhz"]
Aug 01 00:00:00.232243 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.232207    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnlnc\" (UniqueName: \"kubernetes.io/projected/eca07b31-b4fa-44f7-9c31-db23e16be4c5-kube-api-access-nnlnc\") pod \"image-pruner-29759040-jszhz\" (UID: \"eca07b31-b4fa-44f7-9c31-db23e16be4c5\") " pod="openshift-image-registry/image-pruner-29759040-jszhz"
Aug 01 00:00:00.232424 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.232259    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/eca07b31-b4fa-44f7-9c31-db23e16be4c5-serviceca\") pod \"image-pruner-29759040-jszhz\" (UID: \"eca07b31-b4fa-44f7-9c31-db23e16be4c5\") " pod="openshift-image-registry/image-pruner-29759040-jszhz"
Aug 01 00:00:00.333080 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.333037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnlnc\" (UniqueName: \"kubernetes.io/projected/eca07b31-b4fa-44f7-9c31-db23e16be4c5-kube-api-access-nnlnc\") pod \"image-pruner-29759040-jszhz\" (UID: \"eca07b31-b4fa-44f7-9c31-db23e16be4c5\") " pod="openshift-image-registry/image-pruner-29759040-jszhz"
Aug 01 00:00:00.333080 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.333085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/eca07b31-b4fa-44f7-9c31-db23e16be4c5-serviceca\") pod \"image-pruner-29759040-jszhz\" (UID: \"eca07b31-b4fa-44f7-9c31-db23e16be4c5\") " pod="openshift-image-registry/image-pruner-29759040-jszhz"
Aug 01 00:00:00.333644 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.333624    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/eca07b31-b4fa-44f7-9c31-db23e16be4c5-serviceca\") pod \"image-pruner-29759040-jszhz\" (UID: \"eca07b31-b4fa-44f7-9c31-db23e16be4c5\") " pod="openshift-image-registry/image-pruner-29759040-jszhz"
Aug 01 00:00:00.341868 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.341842    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnlnc\" (UniqueName: \"kubernetes.io/projected/eca07b31-b4fa-44f7-9c31-db23e16be4c5-kube-api-access-nnlnc\") pod \"image-pruner-29759040-jszhz\" (UID: \"eca07b31-b4fa-44f7-9c31-db23e16be4c5\") " pod="openshift-image-registry/image-pruner-29759040-jszhz"
Aug 01 00:00:00.477179 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.477143    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29759040-jszhz"
Aug 01 00:00:00.606462 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:00.606427    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29759040-jszhz"]
Aug 01 00:00:00.609605 ip-10-0-139-141 kubenswrapper[2581]: W0801 00:00:00.609576    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca07b31_b4fa_44f7_9c31_db23e16be4c5.slice/crio-7262d44a16463c3546dd31598dd16aee27cf425c9d5fef3149f6c7087eb304ad WatchSource:0}: Error finding container 7262d44a16463c3546dd31598dd16aee27cf425c9d5fef3149f6c7087eb304ad: Status 404 returned error can't find the container with id 7262d44a16463c3546dd31598dd16aee27cf425c9d5fef3149f6c7087eb304ad
Aug 01 00:00:01.460029 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:01.459987    2581 generic.go:356] "Generic (PLEG): container finished" podID="eca07b31-b4fa-44f7-9c31-db23e16be4c5" containerID="df72277c103cde60603ac5c349f2a5462fb5110e1fa480f99f337eb072d7d144" exitCode=0
Aug 01 00:00:01.460416 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:01.460048    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29759040-jszhz" event={"ID":"eca07b31-b4fa-44f7-9c31-db23e16be4c5","Type":"ContainerDied","Data":"df72277c103cde60603ac5c349f2a5462fb5110e1fa480f99f337eb072d7d144"}
Aug 01 00:00:01.460416 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:01.460102    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29759040-jszhz" event={"ID":"eca07b31-b4fa-44f7-9c31-db23e16be4c5","Type":"ContainerStarted","Data":"7262d44a16463c3546dd31598dd16aee27cf425c9d5fef3149f6c7087eb304ad"}
Aug 01 00:00:02.598066 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:02.598031    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29759040-jszhz"
Aug 01 00:00:02.651554 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:02.651517    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/eca07b31-b4fa-44f7-9c31-db23e16be4c5-serviceca\") pod \"eca07b31-b4fa-44f7-9c31-db23e16be4c5\" (UID: \"eca07b31-b4fa-44f7-9c31-db23e16be4c5\") "
Aug 01 00:00:02.651732 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:02.651585    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnlnc\" (UniqueName: \"kubernetes.io/projected/eca07b31-b4fa-44f7-9c31-db23e16be4c5-kube-api-access-nnlnc\") pod \"eca07b31-b4fa-44f7-9c31-db23e16be4c5\" (UID: \"eca07b31-b4fa-44f7-9c31-db23e16be4c5\") "
Aug 01 00:00:02.652172 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:02.652148    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eca07b31-b4fa-44f7-9c31-db23e16be4c5-serviceca" (OuterVolumeSpecName: "serviceca") pod "eca07b31-b4fa-44f7-9c31-db23e16be4c5" (UID: "eca07b31-b4fa-44f7-9c31-db23e16be4c5"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:00:02.653815 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:02.653793    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eca07b31-b4fa-44f7-9c31-db23e16be4c5-kube-api-access-nnlnc" (OuterVolumeSpecName: "kube-api-access-nnlnc") pod "eca07b31-b4fa-44f7-9c31-db23e16be4c5" (UID: "eca07b31-b4fa-44f7-9c31-db23e16be4c5"). InnerVolumeSpecName "kube-api-access-nnlnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:00:02.752230 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:02.752138    2581 reconciler_common.go:299] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/eca07b31-b4fa-44f7-9c31-db23e16be4c5-serviceca\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:00:02.752230 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:02.752166    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnlnc\" (UniqueName: \"kubernetes.io/projected/eca07b31-b4fa-44f7-9c31-db23e16be4c5-kube-api-access-nnlnc\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:00:03.239843 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:03.239790    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Aug 01 00:00:03.468293 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:03.468261    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29759040-jszhz" event={"ID":"eca07b31-b4fa-44f7-9c31-db23e16be4c5","Type":"ContainerDied","Data":"7262d44a16463c3546dd31598dd16aee27cf425c9d5fef3149f6c7087eb304ad"}
Aug 01 00:00:03.468452 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:03.468314    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7262d44a16463c3546dd31598dd16aee27cf425c9d5fef3149f6c7087eb304ad"
Aug 01 00:00:03.468452 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:03.468271    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29759040-jszhz"
Aug 01 00:00:05.387889 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:05.387851    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Aug 01 00:00:15.388838 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:15.388809    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Aug 01 00:00:30.647298 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:30.647259    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"]
Aug 01 00:00:30.647780 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:30.647534    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca07b31-b4fa-44f7-9c31-db23e16be4c5" containerName="image-pruner"
Aug 01 00:00:30.647780 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:30.647545    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca07b31-b4fa-44f7-9c31-db23e16be4c5" containerName="image-pruner"
Aug 01 00:00:30.647780 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:30.647597    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca07b31-b4fa-44f7-9c31-db23e16be4c5" containerName="image-pruner"
Aug 01 00:00:30.651635 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:30.651615    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"
Aug 01 00:00:30.654143 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:30.654120    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-64006-serving-cert\""
Aug 01 00:00:30.654245 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:30.654202    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-64006-kube-rbac-proxy-sar-config\""
Aug 01 00:00:30.660648 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:30.660621    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"]
Aug 01 00:00:30.768922 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:30.768877    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c803d3-2880-45da-ad17-7965a28ac29a-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-64006-74fb765f4c-mzccz\" (UID: \"59c803d3-2880-45da-ad17-7965a28ac29a\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"
Aug 01 00:00:30.768922 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:30.768928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/59c803d3-2880-45da-ad17-7965a28ac29a-proxy-tls\") pod \"model-chainer-raw-hpa-64006-74fb765f4c-mzccz\" (UID: \"59c803d3-2880-45da-ad17-7965a28ac29a\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"
Aug 01 00:00:30.870102 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:30.870057    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c803d3-2880-45da-ad17-7965a28ac29a-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-64006-74fb765f4c-mzccz\" (UID: \"59c803d3-2880-45da-ad17-7965a28ac29a\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"
Aug 01 00:00:30.870102 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:30.870103    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/59c803d3-2880-45da-ad17-7965a28ac29a-proxy-tls\") pod \"model-chainer-raw-hpa-64006-74fb765f4c-mzccz\" (UID: \"59c803d3-2880-45da-ad17-7965a28ac29a\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"
Aug 01 00:00:30.870741 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:30.870717    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c803d3-2880-45da-ad17-7965a28ac29a-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-64006-74fb765f4c-mzccz\" (UID: \"59c803d3-2880-45da-ad17-7965a28ac29a\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"
Aug 01 00:00:30.872685 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:30.872646    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/59c803d3-2880-45da-ad17-7965a28ac29a-proxy-tls\") pod \"model-chainer-raw-hpa-64006-74fb765f4c-mzccz\" (UID: \"59c803d3-2880-45da-ad17-7965a28ac29a\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"
Aug 01 00:00:30.962884 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:30.962798    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"
Aug 01 00:00:31.092802 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:31.092771    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"]
Aug 01 00:00:31.095341 ip-10-0-139-141 kubenswrapper[2581]: W0801 00:00:31.095314    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c803d3_2880_45da_ad17_7965a28ac29a.slice/crio-284ad91f6e345ca71b43e8b4c207fa3a23f3583f6df33d4e323aeac5dc278adc WatchSource:0}: Error finding container 284ad91f6e345ca71b43e8b4c207fa3a23f3583f6df33d4e323aeac5dc278adc: Status 404 returned error can't find the container with id 284ad91f6e345ca71b43e8b4c207fa3a23f3583f6df33d4e323aeac5dc278adc
Aug 01 00:00:31.568527 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:31.568490    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz" event={"ID":"59c803d3-2880-45da-ad17-7965a28ac29a","Type":"ContainerStarted","Data":"1abfa13c0eff7c04cdf06b77591f88d092291fc26d58bfe9379f5291787f0c33"}
Aug 01 00:00:31.568724 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:31.568551    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz" event={"ID":"59c803d3-2880-45da-ad17-7965a28ac29a","Type":"ContainerStarted","Data":"284ad91f6e345ca71b43e8b4c207fa3a23f3583f6df33d4e323aeac5dc278adc"}
Aug 01 00:00:31.568724 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:31.568592    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"
Aug 01 00:00:31.583669 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:31.583613    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz" podStartSLOduration=1.5835966830000001 podStartE2EDuration="1.583596683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:00:31.582910946 +0000 UTC m=+854.762367690" watchObservedRunningTime="2026-08-01 00:00:31.583596683 +0000 UTC m=+854.763053427"
Aug 01 00:00:37.578340 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:37.578302    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"
Aug 01 00:00:40.707083 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:40.707050    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"]
Aug 01 00:00:40.707509 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:40.707245    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz" podUID="59c803d3-2880-45da-ad17-7965a28ac29a" containerName="model-chainer-raw-hpa-64006" containerID="cri-o://1abfa13c0eff7c04cdf06b77591f88d092291fc26d58bfe9379f5291787f0c33" gracePeriod=30
Aug 01 00:00:40.835816 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:40.835782    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"]
Aug 01 00:00:40.836067 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:40.836038    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kserve-container" containerID="cri-o://35b48e4266a8bed8ada8ada8ada425947d9012ba465bd062e713d340842ee03a" gracePeriod=30
Aug 01 00:00:40.836169 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:40.836087    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kube-rbac-proxy" containerID="cri-o://ebb96a2f15a8c1b14f8c8eed0dd63d428d792a2b53678cfd63db0d3ee249e405" gracePeriod=30
Aug 01 00:00:41.606278 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:41.606244    2581 generic.go:356] "Generic (PLEG): container finished" podID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerID="ebb96a2f15a8c1b14f8c8eed0dd63d428d792a2b53678cfd63db0d3ee249e405" exitCode=2
Aug 01 00:00:41.606428 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:41.606317    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" event={"ID":"7aeed7fb-652f-4ea1-adf7-442ba749345d","Type":"ContainerDied","Data":"ebb96a2f15a8c1b14f8c8eed0dd63d428d792a2b53678cfd63db0d3ee249e405"}
Aug 01 00:00:42.577469 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:42.577425    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz" podUID="59c803d3-2880-45da-ad17-7965a28ac29a" containerName="model-chainer-raw-hpa-64006" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:00:43.234786 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:43.234736    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused"
Aug 01 00:00:45.387810 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:45.387772    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Aug 01 00:00:45.893999 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:45.893969    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Aug 01 00:00:46.003700 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.003605    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcxbl\" (UniqueName: \"kubernetes.io/projected/7aeed7fb-652f-4ea1-adf7-442ba749345d-kube-api-access-gcxbl\") pod \"7aeed7fb-652f-4ea1-adf7-442ba749345d\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") "
Aug 01 00:00:46.003700 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.003651    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7aeed7fb-652f-4ea1-adf7-442ba749345d-kube-rbac-proxy-sar-config\") pod \"7aeed7fb-652f-4ea1-adf7-442ba749345d\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") "
Aug 01 00:00:46.003700 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.003683    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7aeed7fb-652f-4ea1-adf7-442ba749345d-proxy-tls\") pod \"7aeed7fb-652f-4ea1-adf7-442ba749345d\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") "
Aug 01 00:00:46.004023 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.003735    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7aeed7fb-652f-4ea1-adf7-442ba749345d-cabundle-cert\") pod \"7aeed7fb-652f-4ea1-adf7-442ba749345d\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") "
Aug 01 00:00:46.004023 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.003759    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7aeed7fb-652f-4ea1-adf7-442ba749345d-kserve-provision-location\") pod \"7aeed7fb-652f-4ea1-adf7-442ba749345d\" (UID: \"7aeed7fb-652f-4ea1-adf7-442ba749345d\") "
Aug 01 00:00:46.004141 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.004104    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aeed7fb-652f-4ea1-adf7-442ba749345d-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "7aeed7fb-652f-4ea1-adf7-442ba749345d" (UID: "7aeed7fb-652f-4ea1-adf7-442ba749345d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:00:46.004141 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.004102    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aeed7fb-652f-4ea1-adf7-442ba749345d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "7aeed7fb-652f-4ea1-adf7-442ba749345d" (UID: "7aeed7fb-652f-4ea1-adf7-442ba749345d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 01 00:00:46.004218 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.004140    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aeed7fb-652f-4ea1-adf7-442ba749345d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "7aeed7fb-652f-4ea1-adf7-442ba749345d" (UID: "7aeed7fb-652f-4ea1-adf7-442ba749345d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:00:46.005798 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.005770    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aeed7fb-652f-4ea1-adf7-442ba749345d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7aeed7fb-652f-4ea1-adf7-442ba749345d" (UID: "7aeed7fb-652f-4ea1-adf7-442ba749345d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 01 00:00:46.005940 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.005894    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aeed7fb-652f-4ea1-adf7-442ba749345d-kube-api-access-gcxbl" (OuterVolumeSpecName: "kube-api-access-gcxbl") pod "7aeed7fb-652f-4ea1-adf7-442ba749345d" (UID: "7aeed7fb-652f-4ea1-adf7-442ba749345d"). InnerVolumeSpecName "kube-api-access-gcxbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:00:46.105260 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.105222    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcxbl\" (UniqueName: \"kubernetes.io/projected/7aeed7fb-652f-4ea1-adf7-442ba749345d-kube-api-access-gcxbl\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:00:46.105260 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.105257    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7aeed7fb-652f-4ea1-adf7-442ba749345d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:00:46.105465 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.105272    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7aeed7fb-652f-4ea1-adf7-442ba749345d-proxy-tls\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:00:46.105465 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.105287    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7aeed7fb-652f-4ea1-adf7-442ba749345d-cabundle-cert\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:00:46.105465 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.105300    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7aeed7fb-652f-4ea1-adf7-442ba749345d-kserve-provision-location\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:00:46.629636 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.629593    2581 generic.go:356] "Generic (PLEG): container finished" podID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerID="35b48e4266a8bed8ada8ada8ada425947d9012ba465bd062e713d340842ee03a" exitCode=0
Aug 01 00:00:46.630166 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.629655    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" event={"ID":"7aeed7fb-652f-4ea1-adf7-442ba749345d","Type":"ContainerDied","Data":"35b48e4266a8bed8ada8ada8ada425947d9012ba465bd062e713d340842ee03a"}
Aug 01 00:00:46.630166 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.629662    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"
Aug 01 00:00:46.630166 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.629708    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l" event={"ID":"7aeed7fb-652f-4ea1-adf7-442ba749345d","Type":"ContainerDied","Data":"ae7102fb197bc82708061a5d7a91d925aba48551d15679e271b5670698191ddb"}
Aug 01 00:00:46.630166 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.629734    2581 scope.go:117] "RemoveContainer" containerID="ebb96a2f15a8c1b14f8c8eed0dd63d428d792a2b53678cfd63db0d3ee249e405"
Aug 01 00:00:46.643905 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.638785    2581 scope.go:117] "RemoveContainer" containerID="35b48e4266a8bed8ada8ada8ada425947d9012ba465bd062e713d340842ee03a"
Aug 01 00:00:46.652226 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.652200    2581 scope.go:117] "RemoveContainer" containerID="c186a8420ed61766cdda61a27675487d7e2f70b425d0726f547a6f6aba5a2c52"
Aug 01 00:00:46.654934 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.654902    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"]
Aug 01 00:00:46.662559 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.662527    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-raw-hpa-64006-predictor-5f79588f4-qnq5l"]
Aug 01 00:00:46.663040 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.663019    2581 scope.go:117] "RemoveContainer" containerID="ebb96a2f15a8c1b14f8c8eed0dd63d428d792a2b53678cfd63db0d3ee249e405"
Aug 01 00:00:46.663361 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:00:46.663340    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebb96a2f15a8c1b14f8c8eed0dd63d428d792a2b53678cfd63db0d3ee249e405\": container with ID starting with ebb96a2f15a8c1b14f8c8eed0dd63d428d792a2b53678cfd63db0d3ee249e405 not found: ID does not exist" containerID="ebb96a2f15a8c1b14f8c8eed0dd63d428d792a2b53678cfd63db0d3ee249e405"
Aug 01 00:00:46.663412 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.663374    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebb96a2f15a8c1b14f8c8eed0dd63d428d792a2b53678cfd63db0d3ee249e405"} err="failed to get container status \"ebb96a2f15a8c1b14f8c8eed0dd63d428d792a2b53678cfd63db0d3ee249e405\": rpc error: code = NotFound desc = could not find container \"ebb96a2f15a8c1b14f8c8eed0dd63d428d792a2b53678cfd63db0d3ee249e405\": container with ID starting with ebb96a2f15a8c1b14f8c8eed0dd63d428d792a2b53678cfd63db0d3ee249e405 not found: ID does not exist"
Aug 01 00:00:46.663412 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.663394    2581 scope.go:117] "RemoveContainer" containerID="35b48e4266a8bed8ada8ada8ada425947d9012ba465bd062e713d340842ee03a"
Aug 01 00:00:46.663649 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:00:46.663631    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35b48e4266a8bed8ada8ada8ada425947d9012ba465bd062e713d340842ee03a\": container with ID starting with 35b48e4266a8bed8ada8ada8ada425947d9012ba465bd062e713d340842ee03a not found: ID does not exist" containerID="35b48e4266a8bed8ada8ada8ada425947d9012ba465bd062e713d340842ee03a"
Aug 01 00:00:46.663724 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.663660    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35b48e4266a8bed8ada8ada8ada425947d9012ba465bd062e713d340842ee03a"} err="failed to get container status \"35b48e4266a8bed8ada8ada8ada425947d9012ba465bd062e713d340842ee03a\": rpc error: code = NotFound desc = could not find container \"35b48e4266a8bed8ada8ada8ada425947d9012ba465bd062e713d340842ee03a\": container with ID starting with 35b48e4266a8bed8ada8ada8ada425947d9012ba465bd062e713d340842ee03a not found: ID does not exist"
Aug 01 00:00:46.663724 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.663685    2581 scope.go:117] "RemoveContainer" containerID="c186a8420ed61766cdda61a27675487d7e2f70b425d0726f547a6f6aba5a2c52"
Aug 01 00:00:46.663967 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:00:46.663938    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c186a8420ed61766cdda61a27675487d7e2f70b425d0726f547a6f6aba5a2c52\": container with ID starting with c186a8420ed61766cdda61a27675487d7e2f70b425d0726f547a6f6aba5a2c52 not found: ID does not exist" containerID="c186a8420ed61766cdda61a27675487d7e2f70b425d0726f547a6f6aba5a2c52"
Aug 01 00:00:46.664004 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:46.663977    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c186a8420ed61766cdda61a27675487d7e2f70b425d0726f547a6f6aba5a2c52"} err="failed to get container status \"c186a8420ed61766cdda61a27675487d7e2f70b425d0726f547a6f6aba5a2c52\": rpc error: code = NotFound desc = could not find container \"c186a8420ed61766cdda61a27675487d7e2f70b425d0726f547a6f6aba5a2c52\": container with ID starting with c186a8420ed61766cdda61a27675487d7e2f70b425d0726f547a6f6aba5a2c52 not found: ID does not exist"
Aug 01 00:00:47.386948 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:47.386913    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" path="/var/lib/kubelet/pods/7aeed7fb-652f-4ea1-adf7-442ba749345d/volumes"
Aug 01 00:00:47.576161 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:47.576124    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz" podUID="59c803d3-2880-45da-ad17-7965a28ac29a" containerName="model-chainer-raw-hpa-64006" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:00:52.576731 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:52.576688    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz" podUID="59c803d3-2880-45da-ad17-7965a28ac29a" containerName="model-chainer-raw-hpa-64006" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:00:52.577238 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:52.576795    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"
Aug 01 00:00:57.575939 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:00:57.575890    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz" podUID="59c803d3-2880-45da-ad17-7965a28ac29a" containerName="model-chainer-raw-hpa-64006" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:01:00.956396 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:00.956351    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"]
Aug 01 00:01:00.956893 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:00.956767    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kserve-container"
Aug 01 00:01:00.956893 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:00.956787    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kserve-container"
Aug 01 00:01:00.956893 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:00.956805    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="storage-initializer"
Aug 01 00:01:00.956893 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:00.956814    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="storage-initializer"
Aug 01 00:01:00.956893 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:00.956823    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kube-rbac-proxy"
Aug 01 00:01:00.956893 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:00.956831    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kube-rbac-proxy"
Aug 01 00:01:00.956893 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:00.956894    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kserve-container"
Aug 01 00:01:00.957274 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:00.956907    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aeed7fb-652f-4ea1-adf7-442ba749345d" containerName="kube-rbac-proxy"
Aug 01 00:01:00.961609 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:00.961585    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:00.964298 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:00.964249    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Aug 01 00:01:00.964429 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:00.964313    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-raw-40d2c-kube-rbac-proxy-sar-config\""
Aug 01 00:01:00.964626 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:00.964611    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-raw-40d2c-predictor-serving-cert\""
Aug 01 00:01:00.964801 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:00.964789    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Aug 01 00:01:00.964894 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:00.964876    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Aug 01 00:01:00.969759 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:00.969736    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"]
Aug 01 00:01:01.018433 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.018391    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8jn9\" (UniqueName: \"kubernetes.io/projected/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kube-api-access-w8jn9\") pod \"isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") " pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:01.018433 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.018438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") " pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:01.018635 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.018488    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-proxy-tls\") pod \"isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") " pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:01.018635 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.018540    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kserve-provision-location\") pod \"isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") " pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:01.018635 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.018561    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-cabundle-cert\") pod \"isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") " pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:01.119547 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.119504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kserve-provision-location\") pod \"isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") " pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:01.119547 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.119543    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-cabundle-cert\") pod \"isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") " pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:01.119794 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.119566    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8jn9\" (UniqueName: \"kubernetes.io/projected/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kube-api-access-w8jn9\") pod \"isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") " pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:01.119794 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.119598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") " pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:01.119794 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.119626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-proxy-tls\") pod \"isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") " pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:01.120029 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.119996    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kserve-provision-location\") pod \"isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") " pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:01.120342 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.120322    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-cabundle-cert\") pod \"isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") " pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:01.120393 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.120366    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") " pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:01.122076 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.122056    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-proxy-tls\") pod \"isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") " pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:01.127184 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.127164    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8jn9\" (UniqueName: \"kubernetes.io/projected/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kube-api-access-w8jn9\") pod \"isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") " pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:01.272095 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.271989    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:01.404689 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.404661    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"]
Aug 01 00:01:01.407288 ip-10-0-139-141 kubenswrapper[2581]: W0801 00:01:01.407251    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7e7cbaf_0864_40c7_a5ce_e07cf6f6c1b0.slice/crio-e1e3e7af801c87e8685d6bb2e9b74117b94697f0da2cb1cc2e4ffa6c62f08c38 WatchSource:0}: Error finding container e1e3e7af801c87e8685d6bb2e9b74117b94697f0da2cb1cc2e4ffa6c62f08c38: Status 404 returned error can't find the container with id e1e3e7af801c87e8685d6bb2e9b74117b94697f0da2cb1cc2e4ffa6c62f08c38
Aug 01 00:01:01.683309 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.683264    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" event={"ID":"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0","Type":"ContainerStarted","Data":"75e6845ad049f3e43fc9efcb9719af71e69a20bf1afbe4eb384f12e4dbaeedf0"}
Aug 01 00:01:01.683559 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:01.683381    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" event={"ID":"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0","Type":"ContainerStarted","Data":"e1e3e7af801c87e8685d6bb2e9b74117b94697f0da2cb1cc2e4ffa6c62f08c38"}
Aug 01 00:01:02.576497 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:02.576456    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz" podUID="59c803d3-2880-45da-ad17-7965a28ac29a" containerName="model-chainer-raw-hpa-64006" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:01:05.700116 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:05.700026    2581 generic.go:356] "Generic (PLEG): container finished" podID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerID="75e6845ad049f3e43fc9efcb9719af71e69a20bf1afbe4eb384f12e4dbaeedf0" exitCode=0
Aug 01 00:01:05.700116 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:05.700100    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" event={"ID":"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0","Type":"ContainerDied","Data":"75e6845ad049f3e43fc9efcb9719af71e69a20bf1afbe4eb384f12e4dbaeedf0"}
Aug 01 00:01:06.706791 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:06.706750    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" event={"ID":"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0","Type":"ContainerStarted","Data":"7dfa0dd1cb108d0173cd2ff3aef9804843daf7b200cea055e27342f032c5386b"}
Aug 01 00:01:06.707272 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:06.706827    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" event={"ID":"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0","Type":"ContainerStarted","Data":"da79287c5dfd782d88aed85ec444d668fcb1efd2b4584542d51b629ca4c43153"}
Aug 01 00:01:06.707272 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:06.706856    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" event={"ID":"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0","Type":"ContainerStarted","Data":"6f5f6f27ad24bc97fb4b8f6184e3195cc93bd7ef007f3929193dcd383a433acd"}
Aug 01 00:01:06.707272 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:06.707187    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:06.707272 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:06.707255    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:06.707427 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:06.707295    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:06.708872 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:06.708840    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 01 00:01:06.709518 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:06.709493    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:01:06.729905 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:06.729833    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podStartSLOduration=2.816875189 podStartE2EDuration="6.729817942s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.912942753s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:01:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:01:06.727661882 +0000 UTC m=+889.907118627" watchObservedRunningTime="2026-08-01 00:01:06.729817942 +0000 UTC m=+889.909274685"
Aug 01 00:01:07.576401 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:07.576358    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz" podUID="59c803d3-2880-45da-ad17-7965a28ac29a" containerName="model-chainer-raw-hpa-64006" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:01:07.711476 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:07.711435    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 01 00:01:07.711944 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:07.711920    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:01:10.865944 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:10.865918    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"
Aug 01 00:01:10.888237 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:10.888203    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/59c803d3-2880-45da-ad17-7965a28ac29a-proxy-tls\") pod \"59c803d3-2880-45da-ad17-7965a28ac29a\" (UID: \"59c803d3-2880-45da-ad17-7965a28ac29a\") "
Aug 01 00:01:10.888237 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:10.888244    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c803d3-2880-45da-ad17-7965a28ac29a-openshift-service-ca-bundle\") pod \"59c803d3-2880-45da-ad17-7965a28ac29a\" (UID: \"59c803d3-2880-45da-ad17-7965a28ac29a\") "
Aug 01 00:01:10.888656 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:10.888629    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59c803d3-2880-45da-ad17-7965a28ac29a-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "59c803d3-2880-45da-ad17-7965a28ac29a" (UID: "59c803d3-2880-45da-ad17-7965a28ac29a"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:01:10.890468 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:10.890435    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59c803d3-2880-45da-ad17-7965a28ac29a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "59c803d3-2880-45da-ad17-7965a28ac29a" (UID: "59c803d3-2880-45da-ad17-7965a28ac29a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 01 00:01:10.989583 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:10.989489    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/59c803d3-2880-45da-ad17-7965a28ac29a-proxy-tls\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:01:10.989583 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:10.989521    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c803d3-2880-45da-ad17-7965a28ac29a-openshift-service-ca-bundle\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:01:11.728262 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:11.728172    2581 generic.go:356] "Generic (PLEG): container finished" podID="59c803d3-2880-45da-ad17-7965a28ac29a" containerID="1abfa13c0eff7c04cdf06b77591f88d092291fc26d58bfe9379f5291787f0c33" exitCode=0
Aug 01 00:01:11.728262 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:11.728215    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"
Aug 01 00:01:11.728442 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:11.728258    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz" event={"ID":"59c803d3-2880-45da-ad17-7965a28ac29a","Type":"ContainerDied","Data":"1abfa13c0eff7c04cdf06b77591f88d092291fc26d58bfe9379f5291787f0c33"}
Aug 01 00:01:11.728442 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:11.728313    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz" event={"ID":"59c803d3-2880-45da-ad17-7965a28ac29a","Type":"ContainerDied","Data":"284ad91f6e345ca71b43e8b4c207fa3a23f3583f6df33d4e323aeac5dc278adc"}
Aug 01 00:01:11.728442 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:11.728335    2581 scope.go:117] "RemoveContainer" containerID="1abfa13c0eff7c04cdf06b77591f88d092291fc26d58bfe9379f5291787f0c33"
Aug 01 00:01:11.736610 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:11.736592    2581 scope.go:117] "RemoveContainer" containerID="1abfa13c0eff7c04cdf06b77591f88d092291fc26d58bfe9379f5291787f0c33"
Aug 01 00:01:11.736897 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:01:11.736873    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1abfa13c0eff7c04cdf06b77591f88d092291fc26d58bfe9379f5291787f0c33\": container with ID starting with 1abfa13c0eff7c04cdf06b77591f88d092291fc26d58bfe9379f5291787f0c33 not found: ID does not exist" containerID="1abfa13c0eff7c04cdf06b77591f88d092291fc26d58bfe9379f5291787f0c33"
Aug 01 00:01:11.737020 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:11.736904    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1abfa13c0eff7c04cdf06b77591f88d092291fc26d58bfe9379f5291787f0c33"} err="failed to get container status \"1abfa13c0eff7c04cdf06b77591f88d092291fc26d58bfe9379f5291787f0c33\": rpc error: code = NotFound desc = could not find container \"1abfa13c0eff7c04cdf06b77591f88d092291fc26d58bfe9379f5291787f0c33\": container with ID starting with 1abfa13c0eff7c04cdf06b77591f88d092291fc26d58bfe9379f5291787f0c33 not found: ID does not exist"
Aug 01 00:01:11.743887 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:11.743858    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"]
Aug 01 00:01:11.749596 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:11.749559    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-64006-74fb765f4c-mzccz"]
Aug 01 00:01:12.715778 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:12.715746    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:01:12.716483 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:12.716453    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 01 00:01:12.716763 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:12.716739    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:01:13.387872 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:13.387833    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59c803d3-2880-45da-ad17-7965a28ac29a" path="/var/lib/kubelet/pods/59c803d3-2880-45da-ad17-7965a28ac29a/volumes"
Aug 01 00:01:22.717061 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:22.717017    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 01 00:01:22.717501 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:22.717474    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:01:32.717356 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:32.717309    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 01 00:01:32.717871 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:32.717842    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:01:42.717067 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:42.717016    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 01 00:01:42.719657 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:42.717550    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:01:52.717028 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:52.716987    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 01 00:01:52.717530 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:01:52.717403    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:02:02.717191 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:02.717144    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 01 00:02:02.717760 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:02.717562    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:02:12.716583 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:12.716532    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 01 00:02:12.717139 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:12.717003    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:02:17.385399 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:17.385361    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 01 00:02:17.385884 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:17.385852    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:02:27.387104 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:27.387067    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:02:27.387579 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:27.387167    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:02:36.123709 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:36.123676    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"]
Aug 01 00:02:36.124117 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:36.123937    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container" containerID="cri-o://6f5f6f27ad24bc97fb4b8f6184e3195cc93bd7ef007f3929193dcd383a433acd" gracePeriod=30
Aug 01 00:02:36.124117 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:36.123992    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kube-rbac-proxy" containerID="cri-o://da79287c5dfd782d88aed85ec444d668fcb1efd2b4584542d51b629ca4c43153" gracePeriod=30
Aug 01 00:02:36.124117 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:36.123992    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent" containerID="cri-o://7dfa0dd1cb108d0173cd2ff3aef9804843daf7b200cea055e27342f032c5386b" gracePeriod=30
Aug 01 00:02:37.035137 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:37.035099    2581 generic.go:356] "Generic (PLEG): container finished" podID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerID="da79287c5dfd782d88aed85ec444d668fcb1efd2b4584542d51b629ca4c43153" exitCode=2
Aug 01 00:02:37.035315 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:37.035169    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" event={"ID":"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0","Type":"ContainerDied","Data":"da79287c5dfd782d88aed85ec444d668fcb1efd2b4584542d51b629ca4c43153"}
Aug 01 00:02:37.385278 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:37.385236    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 01 00:02:37.385712 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:37.385602    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:02:37.712473 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:37.712378    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Aug 01 00:02:41.052256 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:41.052225    2581 generic.go:356] "Generic (PLEG): container finished" podID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerID="6f5f6f27ad24bc97fb4b8f6184e3195cc93bd7ef007f3929193dcd383a433acd" exitCode=0
Aug 01 00:02:41.052618 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:41.052273    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" event={"ID":"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0","Type":"ContainerDied","Data":"6f5f6f27ad24bc97fb4b8f6184e3195cc93bd7ef007f3929193dcd383a433acd"}
Aug 01 00:02:42.712723 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:42.712673    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Aug 01 00:02:47.385264 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:47.385214    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 01 00:02:47.385852 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:47.385526    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:02:47.712328 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:47.712223    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Aug 01 00:02:47.712523 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:47.712351    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:02:52.712681 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:52.712639    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Aug 01 00:02:57.385313 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:57.385251    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Aug 01 00:02:57.385807 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:57.385610    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:02:57.387597 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:57.387571    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:02:57.387781 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:57.387684    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:02:57.712332 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:02:57.712218    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Aug 01 00:03:02.712743 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:02.712695    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Aug 01 00:03:06.264105 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:06.264072    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:03:06.411467 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:06.411378    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kube-rbac-proxy-sar-config\") pod \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") "
Aug 01 00:03:06.411467 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:06.411414    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-proxy-tls\") pod \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") "
Aug 01 00:03:06.411467 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:06.411439    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-cabundle-cert\") pod \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") "
Aug 01 00:03:06.411467 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:06.411463    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8jn9\" (UniqueName: \"kubernetes.io/projected/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kube-api-access-w8jn9\") pod \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") "
Aug 01 00:03:06.411745 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:06.411585    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kserve-provision-location\") pod \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\" (UID: \"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0\") "
Aug 01 00:03:06.411828 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:06.411800    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" (UID: "b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:03:06.411893 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:06.411810    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" (UID: "b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:03:06.412004 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:06.411983    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" (UID: "b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 01 00:03:06.413582 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:06.413561    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" (UID: "b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 01 00:03:06.413677 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:06.413578    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kube-api-access-w8jn9" (OuterVolumeSpecName: "kube-api-access-w8jn9") pod "b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" (UID: "b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0"). InnerVolumeSpecName "kube-api-access-w8jn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:03:06.512536 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:06.512498    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:03:06.512536 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:06.512528    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-proxy-tls\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:03:06.512536 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:06.512538    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-cabundle-cert\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:03:06.512536 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:06.512547    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8jn9\" (UniqueName: \"kubernetes.io/projected/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kube-api-access-w8jn9\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:03:06.512804 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:06.512556    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0-kserve-provision-location\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:03:07.144729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.144691    2581 generic.go:356] "Generic (PLEG): container finished" podID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerID="7dfa0dd1cb108d0173cd2ff3aef9804843daf7b200cea055e27342f032c5386b" exitCode=0
Aug 01 00:03:07.144897 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.144762    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"
Aug 01 00:03:07.144897 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.144771    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" event={"ID":"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0","Type":"ContainerDied","Data":"7dfa0dd1cb108d0173cd2ff3aef9804843daf7b200cea055e27342f032c5386b"}
Aug 01 00:03:07.144897 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.144840    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5" event={"ID":"b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0","Type":"ContainerDied","Data":"e1e3e7af801c87e8685d6bb2e9b74117b94697f0da2cb1cc2e4ffa6c62f08c38"}
Aug 01 00:03:07.144897 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.144873    2581 scope.go:117] "RemoveContainer" containerID="7dfa0dd1cb108d0173cd2ff3aef9804843daf7b200cea055e27342f032c5386b"
Aug 01 00:03:07.153532 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.153513    2581 scope.go:117] "RemoveContainer" containerID="da79287c5dfd782d88aed85ec444d668fcb1efd2b4584542d51b629ca4c43153"
Aug 01 00:03:07.160793 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.160776    2581 scope.go:117] "RemoveContainer" containerID="6f5f6f27ad24bc97fb4b8f6184e3195cc93bd7ef007f3929193dcd383a433acd"
Aug 01 00:03:07.166107 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.166082    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"]
Aug 01 00:03:07.167859 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.167841    2581 scope.go:117] "RemoveContainer" containerID="75e6845ad049f3e43fc9efcb9719af71e69a20bf1afbe4eb384f12e4dbaeedf0"
Aug 01 00:03:07.170174 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.170139    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-40d2c-predictor-7d9dd75d8d-wfdl5"]
Aug 01 00:03:07.175586 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.175565    2581 scope.go:117] "RemoveContainer" containerID="7dfa0dd1cb108d0173cd2ff3aef9804843daf7b200cea055e27342f032c5386b"
Aug 01 00:03:07.175847 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:03:07.175828    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dfa0dd1cb108d0173cd2ff3aef9804843daf7b200cea055e27342f032c5386b\": container with ID starting with 7dfa0dd1cb108d0173cd2ff3aef9804843daf7b200cea055e27342f032c5386b not found: ID does not exist" containerID="7dfa0dd1cb108d0173cd2ff3aef9804843daf7b200cea055e27342f032c5386b"
Aug 01 00:03:07.175897 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.175857    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dfa0dd1cb108d0173cd2ff3aef9804843daf7b200cea055e27342f032c5386b"} err="failed to get container status \"7dfa0dd1cb108d0173cd2ff3aef9804843daf7b200cea055e27342f032c5386b\": rpc error: code = NotFound desc = could not find container \"7dfa0dd1cb108d0173cd2ff3aef9804843daf7b200cea055e27342f032c5386b\": container with ID starting with 7dfa0dd1cb108d0173cd2ff3aef9804843daf7b200cea055e27342f032c5386b not found: ID does not exist"
Aug 01 00:03:07.175897 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.175875    2581 scope.go:117] "RemoveContainer" containerID="da79287c5dfd782d88aed85ec444d668fcb1efd2b4584542d51b629ca4c43153"
Aug 01 00:03:07.176160 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:03:07.176140    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da79287c5dfd782d88aed85ec444d668fcb1efd2b4584542d51b629ca4c43153\": container with ID starting with da79287c5dfd782d88aed85ec444d668fcb1efd2b4584542d51b629ca4c43153 not found: ID does not exist" containerID="da79287c5dfd782d88aed85ec444d668fcb1efd2b4584542d51b629ca4c43153"
Aug 01 00:03:07.176205 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.176168    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da79287c5dfd782d88aed85ec444d668fcb1efd2b4584542d51b629ca4c43153"} err="failed to get container status \"da79287c5dfd782d88aed85ec444d668fcb1efd2b4584542d51b629ca4c43153\": rpc error: code = NotFound desc = could not find container \"da79287c5dfd782d88aed85ec444d668fcb1efd2b4584542d51b629ca4c43153\": container with ID starting with da79287c5dfd782d88aed85ec444d668fcb1efd2b4584542d51b629ca4c43153 not found: ID does not exist"
Aug 01 00:03:07.176205 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.176185    2581 scope.go:117] "RemoveContainer" containerID="6f5f6f27ad24bc97fb4b8f6184e3195cc93bd7ef007f3929193dcd383a433acd"
Aug 01 00:03:07.176406 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:03:07.176387    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f5f6f27ad24bc97fb4b8f6184e3195cc93bd7ef007f3929193dcd383a433acd\": container with ID starting with 6f5f6f27ad24bc97fb4b8f6184e3195cc93bd7ef007f3929193dcd383a433acd not found: ID does not exist" containerID="6f5f6f27ad24bc97fb4b8f6184e3195cc93bd7ef007f3929193dcd383a433acd"
Aug 01 00:03:07.176451 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.176414    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f5f6f27ad24bc97fb4b8f6184e3195cc93bd7ef007f3929193dcd383a433acd"} err="failed to get container status \"6f5f6f27ad24bc97fb4b8f6184e3195cc93bd7ef007f3929193dcd383a433acd\": rpc error: code = NotFound desc = could not find container \"6f5f6f27ad24bc97fb4b8f6184e3195cc93bd7ef007f3929193dcd383a433acd\": container with ID starting with 6f5f6f27ad24bc97fb4b8f6184e3195cc93bd7ef007f3929193dcd383a433acd not found: ID does not exist"
Aug 01 00:03:07.176451 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.176430    2581 scope.go:117] "RemoveContainer" containerID="75e6845ad049f3e43fc9efcb9719af71e69a20bf1afbe4eb384f12e4dbaeedf0"
Aug 01 00:03:07.176643 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:03:07.176621    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75e6845ad049f3e43fc9efcb9719af71e69a20bf1afbe4eb384f12e4dbaeedf0\": container with ID starting with 75e6845ad049f3e43fc9efcb9719af71e69a20bf1afbe4eb384f12e4dbaeedf0 not found: ID does not exist" containerID="75e6845ad049f3e43fc9efcb9719af71e69a20bf1afbe4eb384f12e4dbaeedf0"
Aug 01 00:03:07.176701 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.176658    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75e6845ad049f3e43fc9efcb9719af71e69a20bf1afbe4eb384f12e4dbaeedf0"} err="failed to get container status \"75e6845ad049f3e43fc9efcb9719af71e69a20bf1afbe4eb384f12e4dbaeedf0\": rpc error: code = NotFound desc = could not find container \"75e6845ad049f3e43fc9efcb9719af71e69a20bf1afbe4eb384f12e4dbaeedf0\": container with ID starting with 75e6845ad049f3e43fc9efcb9719af71e69a20bf1afbe4eb384f12e4dbaeedf0 not found: ID does not exist"
Aug 01 00:03:07.386888 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:03:07.386854    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" path="/var/lib/kubelet/pods/b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0/volumes"
Aug 01 00:04:19.667043 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.666928    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"]
Aug 01 00:04:19.667491 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.667202    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container"
Aug 01 00:04:19.667491 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.667215    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container"
Aug 01 00:04:19.667491 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.667224    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59c803d3-2880-45da-ad17-7965a28ac29a" containerName="model-chainer-raw-hpa-64006"
Aug 01 00:04:19.667491 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.667232    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c803d3-2880-45da-ad17-7965a28ac29a" containerName="model-chainer-raw-hpa-64006"
Aug 01 00:04:19.667491 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.667242    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="storage-initializer"
Aug 01 00:04:19.667491 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.667248    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="storage-initializer"
Aug 01 00:04:19.667491 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.667254    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kube-rbac-proxy"
Aug 01 00:04:19.667491 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.667259    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kube-rbac-proxy"
Aug 01 00:04:19.667491 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.667265    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent"
Aug 01 00:04:19.667491 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.667269    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent"
Aug 01 00:04:19.667491 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.667312    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kserve-container"
Aug 01 00:04:19.667491 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.667319    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="agent"
Aug 01 00:04:19.667491 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.667325    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="59c803d3-2880-45da-ad17-7965a28ac29a" containerName="model-chainer-raw-hpa-64006"
Aug 01 00:04:19.667491 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.667331    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7e7cbaf-0864-40c7-a5ce-e07cf6f6c1b0" containerName="kube-rbac-proxy"
Aug 01 00:04:19.670199 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.670179    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:19.673031 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.673000    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Aug 01 00:04:19.673207 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.673038    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"raw-sklearn-runtime-f36b8-predictor-serving-cert\""
Aug 01 00:04:19.673207 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.673096    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Aug 01 00:04:19.673207 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.673041    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wctjk\""
Aug 01 00:04:19.673207 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.672997    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Aug 01 00:04:19.673918 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.673895    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Aug 01 00:04:19.674084 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.673922    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"raw-sklearn-runtime-f36b8-kube-rbac-proxy-sar-config\""
Aug 01 00:04:19.683545 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.682923    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"]
Aug 01 00:04:19.770694 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.770649    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/eefdb068-8461-4bf8-be31-1265883c2116-kserve-provision-location\") pod \"raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:19.770886 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.770705    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eefdb068-8461-4bf8-be31-1265883c2116-proxy-tls\") pod \"raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:19.770886 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.770729    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vdpm\" (UniqueName: \"kubernetes.io/projected/eefdb068-8461-4bf8-be31-1265883c2116-kube-api-access-7vdpm\") pod \"raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:19.770886 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.770753    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eefdb068-8461-4bf8-be31-1265883c2116-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:19.770886 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.770788    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/eefdb068-8461-4bf8-be31-1265883c2116-cabundle-cert\") pod \"raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:19.871612 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.871557    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/eefdb068-8461-4bf8-be31-1265883c2116-cabundle-cert\") pod \"raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:19.871827 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.871626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/eefdb068-8461-4bf8-be31-1265883c2116-kserve-provision-location\") pod \"raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:19.871827 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.871666    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eefdb068-8461-4bf8-be31-1265883c2116-proxy-tls\") pod \"raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:19.871827 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.871695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vdpm\" (UniqueName: \"kubernetes.io/projected/eefdb068-8461-4bf8-be31-1265883c2116-kube-api-access-7vdpm\") pod \"raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:19.871827 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.871721    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eefdb068-8461-4bf8-be31-1265883c2116-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:19.872153 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.872126    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/eefdb068-8461-4bf8-be31-1265883c2116-kserve-provision-location\") pod \"raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:19.872381 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.872359    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/eefdb068-8461-4bf8-be31-1265883c2116-cabundle-cert\") pod \"raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:19.872458 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.872420    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eefdb068-8461-4bf8-be31-1265883c2116-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:19.874329 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.874303    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eefdb068-8461-4bf8-be31-1265883c2116-proxy-tls\") pod \"raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:19.880736 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.880711    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vdpm\" (UniqueName: \"kubernetes.io/projected/eefdb068-8461-4bf8-be31-1265883c2116-kube-api-access-7vdpm\") pod \"raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") " pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:19.987314 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:19.987221    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:20.124525 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:20.124384    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"]
Aug 01 00:04:20.126986 ip-10-0-139-141 kubenswrapper[2581]: W0801 00:04:20.126936    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeefdb068_8461_4bf8_be31_1265883c2116.slice/crio-fdf44d58f69e643a33cb9768e3c53d1d82ece84a7621f96d1b0f27eeaa033d23 WatchSource:0}: Error finding container fdf44d58f69e643a33cb9768e3c53d1d82ece84a7621f96d1b0f27eeaa033d23: Status 404 returned error can't find the container with id fdf44d58f69e643a33cb9768e3c53d1d82ece84a7621f96d1b0f27eeaa033d23
Aug 01 00:04:20.399562 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:20.399517    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" event={"ID":"eefdb068-8461-4bf8-be31-1265883c2116","Type":"ContainerStarted","Data":"f14caec7c726288a1077e8290437af8f9a47795f61d46af4d8117c53b9f45479"}
Aug 01 00:04:20.399742 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:20.399586    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" event={"ID":"eefdb068-8461-4bf8-be31-1265883c2116","Type":"ContainerStarted","Data":"fdf44d58f69e643a33cb9768e3c53d1d82ece84a7621f96d1b0f27eeaa033d23"}
Aug 01 00:04:24.416656 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:24.416624    2581 generic.go:356] "Generic (PLEG): container finished" podID="eefdb068-8461-4bf8-be31-1265883c2116" containerID="f14caec7c726288a1077e8290437af8f9a47795f61d46af4d8117c53b9f45479" exitCode=0
Aug 01 00:04:24.417053 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:24.416699    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" event={"ID":"eefdb068-8461-4bf8-be31-1265883c2116","Type":"ContainerDied","Data":"f14caec7c726288a1077e8290437af8f9a47795f61d46af4d8117c53b9f45479"}
Aug 01 00:04:25.428214 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:25.428175    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" event={"ID":"eefdb068-8461-4bf8-be31-1265883c2116","Type":"ContainerStarted","Data":"60701897a4369ab82404940354685fda4b790bb848bf8e05fbb624ba0c56857d"}
Aug 01 00:04:25.428688 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:25.428264    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" event={"ID":"eefdb068-8461-4bf8-be31-1265883c2116","Type":"ContainerStarted","Data":"3baf474309f22f83133054e00c5e8fc56602ecd0468e1875f58bc02aaa80dd55"}
Aug 01 00:04:25.428688 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:25.428587    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:25.428786 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:25.428731    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:25.430061 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:25.430033    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 01 00:04:25.449260 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:25.449198    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" podStartSLOduration=2.717686927 podStartE2EDuration="6.44918079s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.731493863s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:04:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:04:25.44709611 +0000 UTC m=+1088.626552864" watchObservedRunningTime="2026-08-01 00:04:25.44918079 +0000 UTC m=+1088.628637533"
Aug 01 00:04:26.431782 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:26.431736    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 01 00:04:31.436944 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:31.436914    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:04:31.437541 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:31.437511    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 01 00:04:41.437772 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:41.437727    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 01 00:04:51.438037 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:04:51.437979    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 01 00:05:01.437728 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:01.437685    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 01 00:05:11.437511 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:11.437453    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 01 00:05:21.437655 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:21.437604    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 01 00:05:31.437831 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.437784    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 01 00:05:31.552577 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.552537    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"]
Aug 01 00:05:31.555854 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.555834    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:31.558275 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.558247    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-create-kube-rbac-proxy-sar-config\""
Aug 01 00:05:31.558564 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.558542    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-create-predictor-serving-cert\""
Aug 01 00:05:31.568654 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.568627    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"]
Aug 01 00:05:31.642125 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.642088    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kserve-provision-location\") pod \"isvc-canary-create-predictor-8644bc947f-zdh6f\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:31.642326 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.642137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-cabundle-cert\") pod \"isvc-canary-create-predictor-8644bc947f-zdh6f\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:31.642326 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.642166    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-proxy-tls\") pod \"isvc-canary-create-predictor-8644bc947f-zdh6f\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:31.642326 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.642199    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nt78\" (UniqueName: \"kubernetes.io/projected/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kube-api-access-2nt78\") pod \"isvc-canary-create-predictor-8644bc947f-zdh6f\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:31.642326 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.642235    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-8644bc947f-zdh6f\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:31.742683 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.742605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-8644bc947f-zdh6f\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:31.742683 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.742669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kserve-provision-location\") pod \"isvc-canary-create-predictor-8644bc947f-zdh6f\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:31.742915 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.742696    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-cabundle-cert\") pod \"isvc-canary-create-predictor-8644bc947f-zdh6f\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:31.742915 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.742720    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-proxy-tls\") pod \"isvc-canary-create-predictor-8644bc947f-zdh6f\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:31.742915 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.742760    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nt78\" (UniqueName: \"kubernetes.io/projected/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kube-api-access-2nt78\") pod \"isvc-canary-create-predictor-8644bc947f-zdh6f\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:31.743173 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.743145    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kserve-provision-location\") pod \"isvc-canary-create-predictor-8644bc947f-zdh6f\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:31.743397 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.743377    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-8644bc947f-zdh6f\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:31.743481 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.743448    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-cabundle-cert\") pod \"isvc-canary-create-predictor-8644bc947f-zdh6f\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:31.745399 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.745374    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-proxy-tls\") pod \"isvc-canary-create-predictor-8644bc947f-zdh6f\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:31.753804 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.753770    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nt78\" (UniqueName: \"kubernetes.io/projected/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kube-api-access-2nt78\") pod \"isvc-canary-create-predictor-8644bc947f-zdh6f\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") " pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:31.865902 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:31.865854    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:32.007198 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:32.007094    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"]
Aug 01 00:05:32.015091 ip-10-0-139-141 kubenswrapper[2581]: W0801 00:05:32.015047    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d8e3222_bc6e_4fdd_a549_db1ac57b617d.slice/crio-64a8bbe5db65b8949139c70eb6fa8a566a18e521d66a8dc321a0c85b216b2bc0 WatchSource:0}: Error finding container 64a8bbe5db65b8949139c70eb6fa8a566a18e521d66a8dc321a0c85b216b2bc0: Status 404 returned error can't find the container with id 64a8bbe5db65b8949139c70eb6fa8a566a18e521d66a8dc321a0c85b216b2bc0
Aug 01 00:05:32.666310 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:32.666269    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" event={"ID":"4d8e3222-bc6e-4fdd-a549-db1ac57b617d","Type":"ContainerStarted","Data":"39231c8178404e0a8eb905a53915bb198309af6e7786be6b92bb786df960b2a0"}
Aug 01 00:05:32.666723 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:32.666371    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" event={"ID":"4d8e3222-bc6e-4fdd-a549-db1ac57b617d","Type":"ContainerStarted","Data":"64a8bbe5db65b8949139c70eb6fa8a566a18e521d66a8dc321a0c85b216b2bc0"}
Aug 01 00:05:36.682759 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:36.682726    2581 generic.go:356] "Generic (PLEG): container finished" podID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerID="39231c8178404e0a8eb905a53915bb198309af6e7786be6b92bb786df960b2a0" exitCode=0
Aug 01 00:05:36.683178 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:36.682811    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" event={"ID":"4d8e3222-bc6e-4fdd-a549-db1ac57b617d","Type":"ContainerDied","Data":"39231c8178404e0a8eb905a53915bb198309af6e7786be6b92bb786df960b2a0"}
Aug 01 00:05:37.688808 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:37.688775    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" event={"ID":"4d8e3222-bc6e-4fdd-a549-db1ac57b617d","Type":"ContainerStarted","Data":"c0bd537d2da428e021685da46ff08004beadd9c48257c7cae82bc0b6f8f059a1"}
Aug 01 00:05:37.689248 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:37.688836    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" event={"ID":"4d8e3222-bc6e-4fdd-a549-db1ac57b617d","Type":"ContainerStarted","Data":"c859d58eabed431308f0533b4e647902c3d9e1fdc0c9f64172fca7fcbcce2299"}
Aug 01 00:05:37.689248 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:37.689099    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:37.689358 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:37.689245    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:37.690557 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:37.690529    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Aug 01 00:05:37.709449 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:37.709396    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" podStartSLOduration=3.087675835 podStartE2EDuration="6.709378069s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.621702234s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:05:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:05:37.707539481 +0000 UTC m=+1160.886996225" watchObservedRunningTime="2026-08-01 00:05:37.709378069 +0000 UTC m=+1160.888834813"
Aug 01 00:05:38.693147 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:38.693113    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Aug 01 00:05:39.384206 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:39.383980    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Aug 01 00:05:43.698029 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:43.697992    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:05:43.698669 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:43.698637    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Aug 01 00:05:49.386710 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:49.386673    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:05:49.795519 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:49.795429    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"]
Aug 01 00:05:49.795719 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:49.795690    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kserve-container" containerID="cri-o://3baf474309f22f83133054e00c5e8fc56602ecd0468e1875f58bc02aaa80dd55" gracePeriod=30
Aug 01 00:05:49.795781 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:49.795719    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kube-rbac-proxy" containerID="cri-o://60701897a4369ab82404940354685fda4b790bb848bf8e05fbb624ba0c56857d" gracePeriod=30
Aug 01 00:05:50.743209 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:50.743172    2581 generic.go:356] "Generic (PLEG): container finished" podID="eefdb068-8461-4bf8-be31-1265883c2116" containerID="60701897a4369ab82404940354685fda4b790bb848bf8e05fbb624ba0c56857d" exitCode=2
Aug 01 00:05:50.743586 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:50.743248    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" event={"ID":"eefdb068-8461-4bf8-be31-1265883c2116","Type":"ContainerDied","Data":"60701897a4369ab82404940354685fda4b790bb848bf8e05fbb624ba0c56857d"}
Aug 01 00:05:51.432682 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:51.432642    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Aug 01 00:05:53.698860 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:53.698814    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Aug 01 00:05:55.032060 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.032035    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:05:55.127502 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.127471    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/eefdb068-8461-4bf8-be31-1265883c2116-cabundle-cert\") pod \"eefdb068-8461-4bf8-be31-1265883c2116\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") "
Aug 01 00:05:55.127653 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.127518    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eefdb068-8461-4bf8-be31-1265883c2116-kube-rbac-proxy-sar-config\") pod \"eefdb068-8461-4bf8-be31-1265883c2116\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") "
Aug 01 00:05:55.127653 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.127557    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eefdb068-8461-4bf8-be31-1265883c2116-proxy-tls\") pod \"eefdb068-8461-4bf8-be31-1265883c2116\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") "
Aug 01 00:05:55.127653 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.127578    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/eefdb068-8461-4bf8-be31-1265883c2116-kserve-provision-location\") pod \"eefdb068-8461-4bf8-be31-1265883c2116\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") "
Aug 01 00:05:55.127653 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.127598    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vdpm\" (UniqueName: \"kubernetes.io/projected/eefdb068-8461-4bf8-be31-1265883c2116-kube-api-access-7vdpm\") pod \"eefdb068-8461-4bf8-be31-1265883c2116\" (UID: \"eefdb068-8461-4bf8-be31-1265883c2116\") "
Aug 01 00:05:55.127943 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.127914    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eefdb068-8461-4bf8-be31-1265883c2116-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "eefdb068-8461-4bf8-be31-1265883c2116" (UID: "eefdb068-8461-4bf8-be31-1265883c2116"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:05:55.128018 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.127917    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eefdb068-8461-4bf8-be31-1265883c2116-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "eefdb068-8461-4bf8-be31-1265883c2116" (UID: "eefdb068-8461-4bf8-be31-1265883c2116"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 01 00:05:55.128018 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.127991    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eefdb068-8461-4bf8-be31-1265883c2116-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "eefdb068-8461-4bf8-be31-1265883c2116" (UID: "eefdb068-8461-4bf8-be31-1265883c2116"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:05:55.129639 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.129616    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eefdb068-8461-4bf8-be31-1265883c2116-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "eefdb068-8461-4bf8-be31-1265883c2116" (UID: "eefdb068-8461-4bf8-be31-1265883c2116"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 01 00:05:55.129742 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.129671    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eefdb068-8461-4bf8-be31-1265883c2116-kube-api-access-7vdpm" (OuterVolumeSpecName: "kube-api-access-7vdpm") pod "eefdb068-8461-4bf8-be31-1265883c2116" (UID: "eefdb068-8461-4bf8-be31-1265883c2116"). InnerVolumeSpecName "kube-api-access-7vdpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:05:55.228523 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.228488    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/eefdb068-8461-4bf8-be31-1265883c2116-cabundle-cert\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:05:55.228523 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.228519    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eefdb068-8461-4bf8-be31-1265883c2116-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:05:55.228523 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.228530    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eefdb068-8461-4bf8-be31-1265883c2116-proxy-tls\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:05:55.228754 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.228539    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/eefdb068-8461-4bf8-be31-1265883c2116-kserve-provision-location\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:05:55.228754 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.228548    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vdpm\" (UniqueName: \"kubernetes.io/projected/eefdb068-8461-4bf8-be31-1265883c2116-kube-api-access-7vdpm\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:05:55.765928 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.765837    2581 generic.go:356] "Generic (PLEG): container finished" podID="eefdb068-8461-4bf8-be31-1265883c2116" containerID="3baf474309f22f83133054e00c5e8fc56602ecd0468e1875f58bc02aaa80dd55" exitCode=0
Aug 01 00:05:55.765928 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.765891    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" event={"ID":"eefdb068-8461-4bf8-be31-1265883c2116","Type":"ContainerDied","Data":"3baf474309f22f83133054e00c5e8fc56602ecd0468e1875f58bc02aaa80dd55"}
Aug 01 00:05:55.765928 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.765902    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"
Aug 01 00:05:55.766195 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.765940    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d" event={"ID":"eefdb068-8461-4bf8-be31-1265883c2116","Type":"ContainerDied","Data":"fdf44d58f69e643a33cb9768e3c53d1d82ece84a7621f96d1b0f27eeaa033d23"}
Aug 01 00:05:55.766195 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.765992    2581 scope.go:117] "RemoveContainer" containerID="60701897a4369ab82404940354685fda4b790bb848bf8e05fbb624ba0c56857d"
Aug 01 00:05:55.774118 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.774093    2581 scope.go:117] "RemoveContainer" containerID="3baf474309f22f83133054e00c5e8fc56602ecd0468e1875f58bc02aaa80dd55"
Aug 01 00:05:55.781548 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.781521    2581 scope.go:117] "RemoveContainer" containerID="f14caec7c726288a1077e8290437af8f9a47795f61d46af4d8117c53b9f45479"
Aug 01 00:05:55.784372 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.784347    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"]
Aug 01 00:05:55.788324 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.788290    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-runtime-f36b8-predictor-6869c565c8-f666d"]
Aug 01 00:05:55.789595 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.789576    2581 scope.go:117] "RemoveContainer" containerID="60701897a4369ab82404940354685fda4b790bb848bf8e05fbb624ba0c56857d"
Aug 01 00:05:55.789858 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:05:55.789838    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60701897a4369ab82404940354685fda4b790bb848bf8e05fbb624ba0c56857d\": container with ID starting with 60701897a4369ab82404940354685fda4b790bb848bf8e05fbb624ba0c56857d not found: ID does not exist" containerID="60701897a4369ab82404940354685fda4b790bb848bf8e05fbb624ba0c56857d"
Aug 01 00:05:55.789914 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.789870    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60701897a4369ab82404940354685fda4b790bb848bf8e05fbb624ba0c56857d"} err="failed to get container status \"60701897a4369ab82404940354685fda4b790bb848bf8e05fbb624ba0c56857d\": rpc error: code = NotFound desc = could not find container \"60701897a4369ab82404940354685fda4b790bb848bf8e05fbb624ba0c56857d\": container with ID starting with 60701897a4369ab82404940354685fda4b790bb848bf8e05fbb624ba0c56857d not found: ID does not exist"
Aug 01 00:05:55.789914 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.789889    2581 scope.go:117] "RemoveContainer" containerID="3baf474309f22f83133054e00c5e8fc56602ecd0468e1875f58bc02aaa80dd55"
Aug 01 00:05:55.790134 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:05:55.790119    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3baf474309f22f83133054e00c5e8fc56602ecd0468e1875f58bc02aaa80dd55\": container with ID starting with 3baf474309f22f83133054e00c5e8fc56602ecd0468e1875f58bc02aaa80dd55 not found: ID does not exist" containerID="3baf474309f22f83133054e00c5e8fc56602ecd0468e1875f58bc02aaa80dd55"
Aug 01 00:05:55.790195 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.790138    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3baf474309f22f83133054e00c5e8fc56602ecd0468e1875f58bc02aaa80dd55"} err="failed to get container status \"3baf474309f22f83133054e00c5e8fc56602ecd0468e1875f58bc02aaa80dd55\": rpc error: code = NotFound desc = could not find container \"3baf474309f22f83133054e00c5e8fc56602ecd0468e1875f58bc02aaa80dd55\": container with ID starting with 3baf474309f22f83133054e00c5e8fc56602ecd0468e1875f58bc02aaa80dd55 not found: ID does not exist"
Aug 01 00:05:55.790195 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.790159    2581 scope.go:117] "RemoveContainer" containerID="f14caec7c726288a1077e8290437af8f9a47795f61d46af4d8117c53b9f45479"
Aug 01 00:05:55.790376 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:05:55.790359    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f14caec7c726288a1077e8290437af8f9a47795f61d46af4d8117c53b9f45479\": container with ID starting with f14caec7c726288a1077e8290437af8f9a47795f61d46af4d8117c53b9f45479 not found: ID does not exist" containerID="f14caec7c726288a1077e8290437af8f9a47795f61d46af4d8117c53b9f45479"
Aug 01 00:05:55.790428 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:55.790385    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f14caec7c726288a1077e8290437af8f9a47795f61d46af4d8117c53b9f45479"} err="failed to get container status \"f14caec7c726288a1077e8290437af8f9a47795f61d46af4d8117c53b9f45479\": rpc error: code = NotFound desc = could not find container \"f14caec7c726288a1077e8290437af8f9a47795f61d46af4d8117c53b9f45479\": container with ID starting with f14caec7c726288a1077e8290437af8f9a47795f61d46af4d8117c53b9f45479 not found: ID does not exist"
Aug 01 00:05:57.386819 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:05:57.386777    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eefdb068-8461-4bf8-be31-1265883c2116" path="/var/lib/kubelet/pods/eefdb068-8461-4bf8-be31-1265883c2116/volumes"
Aug 01 00:06:03.699024 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:06:03.698976    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Aug 01 00:06:13.699076 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:06:13.699031    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Aug 01 00:06:23.699138 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:06:23.699096    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Aug 01 00:06:33.699354 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:06:33.699309    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Aug 01 00:06:43.698578 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:06:43.698533    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Aug 01 00:06:53.699467 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:06:53.699435    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:07:01.617658 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:01.617622    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"]
Aug 01 00:07:01.618112 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:01.617892    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kserve-container" containerID="cri-o://c859d58eabed431308f0533b4e647902c3d9e1fdc0c9f64172fca7fcbcce2299" gracePeriod=30
Aug 01 00:07:01.618112 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:01.617983    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kube-rbac-proxy" containerID="cri-o://c0bd537d2da428e021685da46ff08004beadd9c48257c7cae82bc0b6f8f059a1" gracePeriod=30
Aug 01 00:07:02.016712 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:02.016673    2581 generic.go:356] "Generic (PLEG): container finished" podID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerID="c0bd537d2da428e021685da46ff08004beadd9c48257c7cae82bc0b6f8f059a1" exitCode=2
Aug 01 00:07:02.016890 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:02.016744    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" event={"ID":"4d8e3222-bc6e-4fdd-a549-db1ac57b617d","Type":"ContainerDied","Data":"c0bd537d2da428e021685da46ff08004beadd9c48257c7cae82bc0b6f8f059a1"}
Aug 01 00:07:03.694288 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:03.694250    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.29:8643/healthz\": dial tcp 10.133.0.29:8643: connect: connection refused"
Aug 01 00:07:03.698648 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:03.698619    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Aug 01 00:07:06.855403 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:06.855375    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:07:06.980554 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:06.980463    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kube-rbac-proxy-sar-config\") pod \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") "
Aug 01 00:07:06.980554 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:06.980537    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nt78\" (UniqueName: \"kubernetes.io/projected/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kube-api-access-2nt78\") pod \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") "
Aug 01 00:07:06.980787 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:06.980676    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kserve-provision-location\") pod \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") "
Aug 01 00:07:06.980787 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:06.980717    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-cabundle-cert\") pod \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") "
Aug 01 00:07:06.980787 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:06.980777    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-proxy-tls\") pod \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\" (UID: \"4d8e3222-bc6e-4fdd-a549-db1ac57b617d\") "
Aug 01 00:07:06.980933 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:06.980904    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4d8e3222-bc6e-4fdd-a549-db1ac57b617d" (UID: "4d8e3222-bc6e-4fdd-a549-db1ac57b617d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:07:06.981057 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:06.981028    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4d8e3222-bc6e-4fdd-a549-db1ac57b617d" (UID: "4d8e3222-bc6e-4fdd-a549-db1ac57b617d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 01 00:07:06.981105 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:06.981046    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:07:06.981162 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:06.981133    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "4d8e3222-bc6e-4fdd-a549-db1ac57b617d" (UID: "4d8e3222-bc6e-4fdd-a549-db1ac57b617d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:07:06.982654 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:06.982633    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kube-api-access-2nt78" (OuterVolumeSpecName: "kube-api-access-2nt78") pod "4d8e3222-bc6e-4fdd-a549-db1ac57b617d" (UID: "4d8e3222-bc6e-4fdd-a549-db1ac57b617d"). InnerVolumeSpecName "kube-api-access-2nt78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:07:06.982748 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:06.982720    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4d8e3222-bc6e-4fdd-a549-db1ac57b617d" (UID: "4d8e3222-bc6e-4fdd-a549-db1ac57b617d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 01 00:07:07.037545 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.037508    2581 generic.go:356] "Generic (PLEG): container finished" podID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerID="c859d58eabed431308f0533b4e647902c3d9e1fdc0c9f64172fca7fcbcce2299" exitCode=0
Aug 01 00:07:07.037720 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.037569    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"
Aug 01 00:07:07.037720 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.037578    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" event={"ID":"4d8e3222-bc6e-4fdd-a549-db1ac57b617d","Type":"ContainerDied","Data":"c859d58eabed431308f0533b4e647902c3d9e1fdc0c9f64172fca7fcbcce2299"}
Aug 01 00:07:07.037720 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.037657    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f" event={"ID":"4d8e3222-bc6e-4fdd-a549-db1ac57b617d","Type":"ContainerDied","Data":"64a8bbe5db65b8949139c70eb6fa8a566a18e521d66a8dc321a0c85b216b2bc0"}
Aug 01 00:07:07.037720 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.037685    2581 scope.go:117] "RemoveContainer" containerID="c0bd537d2da428e021685da46ff08004beadd9c48257c7cae82bc0b6f8f059a1"
Aug 01 00:07:07.046823 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.046804    2581 scope.go:117] "RemoveContainer" containerID="c859d58eabed431308f0533b4e647902c3d9e1fdc0c9f64172fca7fcbcce2299"
Aug 01 00:07:07.054610 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.054589    2581 scope.go:117] "RemoveContainer" containerID="39231c8178404e0a8eb905a53915bb198309af6e7786be6b92bb786df960b2a0"
Aug 01 00:07:07.062557 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.062523    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"]
Aug 01 00:07:07.063422 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.063407    2581 scope.go:117] "RemoveContainer" containerID="c0bd537d2da428e021685da46ff08004beadd9c48257c7cae82bc0b6f8f059a1"
Aug 01 00:07:07.063713 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:07:07.063694    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0bd537d2da428e021685da46ff08004beadd9c48257c7cae82bc0b6f8f059a1\": container with ID starting with c0bd537d2da428e021685da46ff08004beadd9c48257c7cae82bc0b6f8f059a1 not found: ID does not exist" containerID="c0bd537d2da428e021685da46ff08004beadd9c48257c7cae82bc0b6f8f059a1"
Aug 01 00:07:07.063758 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.063725    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0bd537d2da428e021685da46ff08004beadd9c48257c7cae82bc0b6f8f059a1"} err="failed to get container status \"c0bd537d2da428e021685da46ff08004beadd9c48257c7cae82bc0b6f8f059a1\": rpc error: code = NotFound desc = could not find container \"c0bd537d2da428e021685da46ff08004beadd9c48257c7cae82bc0b6f8f059a1\": container with ID starting with c0bd537d2da428e021685da46ff08004beadd9c48257c7cae82bc0b6f8f059a1 not found: ID does not exist"
Aug 01 00:07:07.063758 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.063744    2581 scope.go:117] "RemoveContainer" containerID="c859d58eabed431308f0533b4e647902c3d9e1fdc0c9f64172fca7fcbcce2299"
Aug 01 00:07:07.064018 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:07:07.064000    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c859d58eabed431308f0533b4e647902c3d9e1fdc0c9f64172fca7fcbcce2299\": container with ID starting with c859d58eabed431308f0533b4e647902c3d9e1fdc0c9f64172fca7fcbcce2299 not found: ID does not exist" containerID="c859d58eabed431308f0533b4e647902c3d9e1fdc0c9f64172fca7fcbcce2299"
Aug 01 00:07:07.064066 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.064023    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c859d58eabed431308f0533b4e647902c3d9e1fdc0c9f64172fca7fcbcce2299"} err="failed to get container status \"c859d58eabed431308f0533b4e647902c3d9e1fdc0c9f64172fca7fcbcce2299\": rpc error: code = NotFound desc = could not find container \"c859d58eabed431308f0533b4e647902c3d9e1fdc0c9f64172fca7fcbcce2299\": container with ID starting with c859d58eabed431308f0533b4e647902c3d9e1fdc0c9f64172fca7fcbcce2299 not found: ID does not exist"
Aug 01 00:07:07.064066 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.064040    2581 scope.go:117] "RemoveContainer" containerID="39231c8178404e0a8eb905a53915bb198309af6e7786be6b92bb786df960b2a0"
Aug 01 00:07:07.064243 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:07:07.064228    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39231c8178404e0a8eb905a53915bb198309af6e7786be6b92bb786df960b2a0\": container with ID starting with 39231c8178404e0a8eb905a53915bb198309af6e7786be6b92bb786df960b2a0 not found: ID does not exist" containerID="39231c8178404e0a8eb905a53915bb198309af6e7786be6b92bb786df960b2a0"
Aug 01 00:07:07.064287 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.064248    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39231c8178404e0a8eb905a53915bb198309af6e7786be6b92bb786df960b2a0"} err="failed to get container status \"39231c8178404e0a8eb905a53915bb198309af6e7786be6b92bb786df960b2a0\": rpc error: code = NotFound desc = could not find container \"39231c8178404e0a8eb905a53915bb198309af6e7786be6b92bb786df960b2a0\": container with ID starting with 39231c8178404e0a8eb905a53915bb198309af6e7786be6b92bb786df960b2a0 not found: ID does not exist"
Aug 01 00:07:07.067215 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.067188    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-create-predictor-8644bc947f-zdh6f"]
Aug 01 00:07:07.082137 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.082105    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-proxy-tls\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:07:07.082137 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.082135    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nt78\" (UniqueName: \"kubernetes.io/projected/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kube-api-access-2nt78\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:07:07.082267 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.082146    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-kserve-provision-location\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:07:07.082267 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.082156    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4d8e3222-bc6e-4fdd-a549-db1ac57b617d-cabundle-cert\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:07:07.387526 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:07:07.387495    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" path="/var/lib/kubelet/pods/4d8e3222-bc6e-4fdd-a549-db1ac57b617d/volumes"
Aug 01 00:08:36.974223 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974186    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"]
Aug 01 00:08:36.974729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974457    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="storage-initializer"
Aug 01 00:08:36.974729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974469    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="storage-initializer"
Aug 01 00:08:36.974729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974478    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kserve-container"
Aug 01 00:08:36.974729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974484    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kserve-container"
Aug 01 00:08:36.974729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974490    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="storage-initializer"
Aug 01 00:08:36.974729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974495    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="storage-initializer"
Aug 01 00:08:36.974729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974505    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kserve-container"
Aug 01 00:08:36.974729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974510    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kserve-container"
Aug 01 00:08:36.974729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974515    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kube-rbac-proxy"
Aug 01 00:08:36.974729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974520    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kube-rbac-proxy"
Aug 01 00:08:36.974729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974528    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kube-rbac-proxy"
Aug 01 00:08:36.974729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974533    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kube-rbac-proxy"
Aug 01 00:08:36.974729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974577    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kube-rbac-proxy"
Aug 01 00:08:36.974729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974586    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d8e3222-bc6e-4fdd-a549-db1ac57b617d" containerName="kserve-container"
Aug 01 00:08:36.974729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974594    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kube-rbac-proxy"
Aug 01 00:08:36.974729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.974600    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="eefdb068-8461-4bf8-be31-1265883c2116" containerName="kserve-container"
Aug 01 00:08:36.977397 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.977380    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:36.980309 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.980274    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Aug 01 00:08:36.980469 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.980274    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Aug 01 00:08:36.980469 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.980278    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Aug 01 00:08:36.980469 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.980440    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\""
Aug 01 00:08:36.980645 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.980506    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Aug 01 00:08:36.981738 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.981721    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-v2-predictor-serving-cert\""
Aug 01 00:08:36.981827 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.981725    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wctjk\""
Aug 01 00:08:36.987732 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:36.987709    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"]
Aug 01 00:08:37.160147 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.160107    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/10aeb518-12ec-4626-b134-7cbeebdabad5-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:37.160327 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.160171    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2kb9\" (UniqueName: \"kubernetes.io/projected/10aeb518-12ec-4626-b134-7cbeebdabad5-kube-api-access-m2kb9\") pod \"isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:37.160327 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.160223    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/10aeb518-12ec-4626-b134-7cbeebdabad5-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:37.160327 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.160248    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/10aeb518-12ec-4626-b134-7cbeebdabad5-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:37.160327 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.160278    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/10aeb518-12ec-4626-b134-7cbeebdabad5-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:37.261065 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.260971    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/10aeb518-12ec-4626-b134-7cbeebdabad5-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:37.261065 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.261044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2kb9\" (UniqueName: \"kubernetes.io/projected/10aeb518-12ec-4626-b134-7cbeebdabad5-kube-api-access-m2kb9\") pod \"isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:37.261293 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.261070    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/10aeb518-12ec-4626-b134-7cbeebdabad5-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:37.261293 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.261093    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/10aeb518-12ec-4626-b134-7cbeebdabad5-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:37.261293 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.261127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/10aeb518-12ec-4626-b134-7cbeebdabad5-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:37.261501 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.261360    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/10aeb518-12ec-4626-b134-7cbeebdabad5-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:37.261768 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.261745    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/10aeb518-12ec-4626-b134-7cbeebdabad5-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:37.261831 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.261754    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/10aeb518-12ec-4626-b134-7cbeebdabad5-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:37.263593 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.263576    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/10aeb518-12ec-4626-b134-7cbeebdabad5-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:37.275546 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.275514    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2kb9\" (UniqueName: \"kubernetes.io/projected/10aeb518-12ec-4626-b134-7cbeebdabad5-kube-api-access-m2kb9\") pod \"isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:37.288410 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.288374    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:37.424742 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:37.424711    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"]
Aug 01 00:08:37.427215 ip-10-0-139-141 kubenswrapper[2581]: W0801 00:08:37.427182    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10aeb518_12ec_4626_b134_7cbeebdabad5.slice/crio-92dbef684cb70c3b0ffdc6928664a5a6e0e1854e7906f3b458189df0def28a7e WatchSource:0}: Error finding container 92dbef684cb70c3b0ffdc6928664a5a6e0e1854e7906f3b458189df0def28a7e: Status 404 returned error can't find the container with id 92dbef684cb70c3b0ffdc6928664a5a6e0e1854e7906f3b458189df0def28a7e
Aug 01 00:08:38.359227 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:38.359184    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" event={"ID":"10aeb518-12ec-4626-b134-7cbeebdabad5","Type":"ContainerStarted","Data":"36b1e3627c6212cffdd00cf7bedd53dfe1e4590d31a77f8bd31d5476e441a628"}
Aug 01 00:08:38.359641 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:38.359251    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" event={"ID":"10aeb518-12ec-4626-b134-7cbeebdabad5","Type":"ContainerStarted","Data":"92dbef684cb70c3b0ffdc6928664a5a6e0e1854e7906f3b458189df0def28a7e"}
Aug 01 00:08:41.372151 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:41.372059    2581 generic.go:356] "Generic (PLEG): container finished" podID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerID="36b1e3627c6212cffdd00cf7bedd53dfe1e4590d31a77f8bd31d5476e441a628" exitCode=0
Aug 01 00:08:41.372611 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:41.372144    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" event={"ID":"10aeb518-12ec-4626-b134-7cbeebdabad5","Type":"ContainerDied","Data":"36b1e3627c6212cffdd00cf7bedd53dfe1e4590d31a77f8bd31d5476e441a628"}
Aug 01 00:08:42.377752 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:42.377714    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" event={"ID":"10aeb518-12ec-4626-b134-7cbeebdabad5","Type":"ContainerStarted","Data":"499f5a73651f55e3b60524a1ee5fc9a54400e558272219fcbd3a49412fb7ded4"}
Aug 01 00:08:42.378211 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:42.377778    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" event={"ID":"10aeb518-12ec-4626-b134-7cbeebdabad5","Type":"ContainerStarted","Data":"198e4cf9e4631ff84582ef50c8bd50a592d7bfec655ac847db67851faae19c20"}
Aug 01 00:08:42.378211 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:42.378069    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:42.378333 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:42.378224    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:42.379499 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:42.379470    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 01 00:08:42.397971 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:42.397914    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" podStartSLOduration=2.947705672 podStartE2EDuration="6.397895803s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.450190131s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:08:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:08:42.395783529 +0000 UTC m=+1345.575240273" watchObservedRunningTime="2026-08-01 00:08:42.397895803 +0000 UTC m=+1345.577352608"
Aug 01 00:08:43.381547 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:43.381503    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 01 00:08:48.386737 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:48.386706    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:08:48.387394 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:48.387363    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 01 00:08:58.388098 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:08:58.388052    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 01 00:09:08.387639 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:09:08.387597    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 01 00:09:18.387760 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:09:18.387717    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 01 00:09:28.387814 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:09:28.387774    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 01 00:09:38.388073 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:09:38.388002    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 01 00:09:48.388019 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:09:48.387979    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 01 00:09:58.388130 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:09:58.388093    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:10:07.137513 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:07.137468    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"]
Aug 01 00:10:07.138028 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:07.137791    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kserve-container" containerID="cri-o://198e4cf9e4631ff84582ef50c8bd50a592d7bfec655ac847db67851faae19c20" gracePeriod=30
Aug 01 00:10:07.138028 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:07.137812    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kube-rbac-proxy" containerID="cri-o://499f5a73651f55e3b60524a1ee5fc9a54400e558272219fcbd3a49412fb7ded4" gracePeriod=30
Aug 01 00:10:07.682234 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:07.682200    2581 generic.go:356] "Generic (PLEG): container finished" podID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerID="499f5a73651f55e3b60524a1ee5fc9a54400e558272219fcbd3a49412fb7ded4" exitCode=2
Aug 01 00:10:07.682420 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:07.682269    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" event={"ID":"10aeb518-12ec-4626-b134-7cbeebdabad5","Type":"ContainerDied","Data":"499f5a73651f55e3b60524a1ee5fc9a54400e558272219fcbd3a49412fb7ded4"}
Aug 01 00:10:08.381939 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:08.381895    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.30:8643/healthz\": dial tcp 10.133.0.30:8643: connect: connection refused"
Aug 01 00:10:08.387345 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:08.387317    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Aug 01 00:10:12.087108 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.087081    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:10:12.189925 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.189888    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/10aeb518-12ec-4626-b134-7cbeebdabad5-kserve-provision-location\") pod \"10aeb518-12ec-4626-b134-7cbeebdabad5\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") "
Aug 01 00:10:12.189925 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.189927    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2kb9\" (UniqueName: \"kubernetes.io/projected/10aeb518-12ec-4626-b134-7cbeebdabad5-kube-api-access-m2kb9\") pod \"10aeb518-12ec-4626-b134-7cbeebdabad5\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") "
Aug 01 00:10:12.190223 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.189980    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/10aeb518-12ec-4626-b134-7cbeebdabad5-proxy-tls\") pod \"10aeb518-12ec-4626-b134-7cbeebdabad5\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") "
Aug 01 00:10:12.190223 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.190009    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/10aeb518-12ec-4626-b134-7cbeebdabad5-cabundle-cert\") pod \"10aeb518-12ec-4626-b134-7cbeebdabad5\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") "
Aug 01 00:10:12.190223 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.190027    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/10aeb518-12ec-4626-b134-7cbeebdabad5-kube-rbac-proxy-sar-config\") pod \"10aeb518-12ec-4626-b134-7cbeebdabad5\" (UID: \"10aeb518-12ec-4626-b134-7cbeebdabad5\") "
Aug 01 00:10:12.190380 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.190291    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10aeb518-12ec-4626-b134-7cbeebdabad5-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "10aeb518-12ec-4626-b134-7cbeebdabad5" (UID: "10aeb518-12ec-4626-b134-7cbeebdabad5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 01 00:10:12.190439 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.190418    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10aeb518-12ec-4626-b134-7cbeebdabad5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "10aeb518-12ec-4626-b134-7cbeebdabad5" (UID: "10aeb518-12ec-4626-b134-7cbeebdabad5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:10:12.190483 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.190417    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10aeb518-12ec-4626-b134-7cbeebdabad5-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "10aeb518-12ec-4626-b134-7cbeebdabad5" (UID: "10aeb518-12ec-4626-b134-7cbeebdabad5"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:10:12.192192 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.192134    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10aeb518-12ec-4626-b134-7cbeebdabad5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "10aeb518-12ec-4626-b134-7cbeebdabad5" (UID: "10aeb518-12ec-4626-b134-7cbeebdabad5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 01 00:10:12.192288 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.192206    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10aeb518-12ec-4626-b134-7cbeebdabad5-kube-api-access-m2kb9" (OuterVolumeSpecName: "kube-api-access-m2kb9") pod "10aeb518-12ec-4626-b134-7cbeebdabad5" (UID: "10aeb518-12ec-4626-b134-7cbeebdabad5"). InnerVolumeSpecName "kube-api-access-m2kb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:10:12.290451 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.290413    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/10aeb518-12ec-4626-b134-7cbeebdabad5-kserve-provision-location\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:10:12.290451 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.290443    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2kb9\" (UniqueName: \"kubernetes.io/projected/10aeb518-12ec-4626-b134-7cbeebdabad5-kube-api-access-m2kb9\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:10:12.290451 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.290454    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/10aeb518-12ec-4626-b134-7cbeebdabad5-proxy-tls\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:10:12.290451 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.290464    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/10aeb518-12ec-4626-b134-7cbeebdabad5-cabundle-cert\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:10:12.290711 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.290474    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/10aeb518-12ec-4626-b134-7cbeebdabad5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:10:12.704888 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.704858    2581 generic.go:356] "Generic (PLEG): container finished" podID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerID="198e4cf9e4631ff84582ef50c8bd50a592d7bfec655ac847db67851faae19c20" exitCode=0
Aug 01 00:10:12.705077 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.704914    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"
Aug 01 00:10:12.705077 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.704941    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" event={"ID":"10aeb518-12ec-4626-b134-7cbeebdabad5","Type":"ContainerDied","Data":"198e4cf9e4631ff84582ef50c8bd50a592d7bfec655ac847db67851faae19c20"}
Aug 01 00:10:12.705077 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.705021    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn" event={"ID":"10aeb518-12ec-4626-b134-7cbeebdabad5","Type":"ContainerDied","Data":"92dbef684cb70c3b0ffdc6928664a5a6e0e1854e7906f3b458189df0def28a7e"}
Aug 01 00:10:12.705077 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.705049    2581 scope.go:117] "RemoveContainer" containerID="499f5a73651f55e3b60524a1ee5fc9a54400e558272219fcbd3a49412fb7ded4"
Aug 01 00:10:12.713010 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.712990    2581 scope.go:117] "RemoveContainer" containerID="198e4cf9e4631ff84582ef50c8bd50a592d7bfec655ac847db67851faae19c20"
Aug 01 00:10:12.722134 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.722112    2581 scope.go:117] "RemoveContainer" containerID="36b1e3627c6212cffdd00cf7bedd53dfe1e4590d31a77f8bd31d5476e441a628"
Aug 01 00:10:12.729130 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.729107    2581 scope.go:117] "RemoveContainer" containerID="499f5a73651f55e3b60524a1ee5fc9a54400e558272219fcbd3a49412fb7ded4"
Aug 01 00:10:12.729390 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:10:12.729371    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"499f5a73651f55e3b60524a1ee5fc9a54400e558272219fcbd3a49412fb7ded4\": container with ID starting with 499f5a73651f55e3b60524a1ee5fc9a54400e558272219fcbd3a49412fb7ded4 not found: ID does not exist" containerID="499f5a73651f55e3b60524a1ee5fc9a54400e558272219fcbd3a49412fb7ded4"
Aug 01 00:10:12.729455 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.729398    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"499f5a73651f55e3b60524a1ee5fc9a54400e558272219fcbd3a49412fb7ded4"} err="failed to get container status \"499f5a73651f55e3b60524a1ee5fc9a54400e558272219fcbd3a49412fb7ded4\": rpc error: code = NotFound desc = could not find container \"499f5a73651f55e3b60524a1ee5fc9a54400e558272219fcbd3a49412fb7ded4\": container with ID starting with 499f5a73651f55e3b60524a1ee5fc9a54400e558272219fcbd3a49412fb7ded4 not found: ID does not exist"
Aug 01 00:10:12.729455 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.729417    2581 scope.go:117] "RemoveContainer" containerID="198e4cf9e4631ff84582ef50c8bd50a592d7bfec655ac847db67851faae19c20"
Aug 01 00:10:12.729648 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:10:12.729629    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"198e4cf9e4631ff84582ef50c8bd50a592d7bfec655ac847db67851faae19c20\": container with ID starting with 198e4cf9e4631ff84582ef50c8bd50a592d7bfec655ac847db67851faae19c20 not found: ID does not exist" containerID="198e4cf9e4631ff84582ef50c8bd50a592d7bfec655ac847db67851faae19c20"
Aug 01 00:10:12.729686 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.729657    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"198e4cf9e4631ff84582ef50c8bd50a592d7bfec655ac847db67851faae19c20"} err="failed to get container status \"198e4cf9e4631ff84582ef50c8bd50a592d7bfec655ac847db67851faae19c20\": rpc error: code = NotFound desc = could not find container \"198e4cf9e4631ff84582ef50c8bd50a592d7bfec655ac847db67851faae19c20\": container with ID starting with 198e4cf9e4631ff84582ef50c8bd50a592d7bfec655ac847db67851faae19c20 not found: ID does not exist"
Aug 01 00:10:12.729686 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.729673    2581 scope.go:117] "RemoveContainer" containerID="36b1e3627c6212cffdd00cf7bedd53dfe1e4590d31a77f8bd31d5476e441a628"
Aug 01 00:10:12.729908 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:10:12.729891    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36b1e3627c6212cffdd00cf7bedd53dfe1e4590d31a77f8bd31d5476e441a628\": container with ID starting with 36b1e3627c6212cffdd00cf7bedd53dfe1e4590d31a77f8bd31d5476e441a628 not found: ID does not exist" containerID="36b1e3627c6212cffdd00cf7bedd53dfe1e4590d31a77f8bd31d5476e441a628"
Aug 01 00:10:12.729972 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.729911    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36b1e3627c6212cffdd00cf7bedd53dfe1e4590d31a77f8bd31d5476e441a628"} err="failed to get container status \"36b1e3627c6212cffdd00cf7bedd53dfe1e4590d31a77f8bd31d5476e441a628\": rpc error: code = NotFound desc = could not find container \"36b1e3627c6212cffdd00cf7bedd53dfe1e4590d31a77f8bd31d5476e441a628\": container with ID starting with 36b1e3627c6212cffdd00cf7bedd53dfe1e4590d31a77f8bd31d5476e441a628 not found: ID does not exist"
Aug 01 00:10:12.751778 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.751744    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"]
Aug 01 00:10:12.757460 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:12.757432    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-7b47596d4d-cmpkn"]
Aug 01 00:10:13.387493 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:13.387464    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" path="/var/lib/kubelet/pods/10aeb518-12ec-4626-b134-7cbeebdabad5/volumes"
Aug 01 00:10:31.261249 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.261205    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"]
Aug 01 00:10:31.261835 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.261794    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="storage-initializer"
Aug 01 00:10:31.261835 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.261813    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="storage-initializer"
Aug 01 00:10:31.261951 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.261840    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kserve-container"
Aug 01 00:10:31.261951 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.261850    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kserve-container"
Aug 01 00:10:31.261951 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.261865    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kube-rbac-proxy"
Aug 01 00:10:31.261951 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.261875    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kube-rbac-proxy"
Aug 01 00:10:31.262180 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.261988    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kserve-container"
Aug 01 00:10:31.262180 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.262002    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="10aeb518-12ec-4626-b134-7cbeebdabad5" containerName="kube-rbac-proxy"
Aug 01 00:10:31.267454 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.267429    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.273788 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.273468    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Aug 01 00:10:31.273788 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.273553    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-raw-cipn-kube-rbac-proxy-sar-config\""
Aug 01 00:10:31.273788 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.273556    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Aug 01 00:10:31.273949 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.273860    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Aug 01 00:10:31.273949 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.273862    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-raw-cipn-predictor-serving-cert\""
Aug 01 00:10:31.274380 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.274359    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wctjk\""
Aug 01 00:10:31.274498 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.274380    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Aug 01 00:10:31.290071 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.290047    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"]
Aug 01 00:10:31.338260 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.338217    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/55645792-0876-4299-acb7-665d6cabb951-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.338441 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.338270    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/55645792-0876-4299-acb7-665d6cabb951-kserve-provision-location\") pod \"isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.338441 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.338297    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fddg\" (UniqueName: \"kubernetes.io/projected/55645792-0876-4299-acb7-665d6cabb951-kube-api-access-8fddg\") pod \"isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.338441 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.338339    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/55645792-0876-4299-acb7-665d6cabb951-cabundle-cert\") pod \"isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.338441 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.338363    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/55645792-0876-4299-acb7-665d6cabb951-proxy-tls\") pod \"isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.439559 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.439517    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fddg\" (UniqueName: \"kubernetes.io/projected/55645792-0876-4299-acb7-665d6cabb951-kube-api-access-8fddg\") pod \"isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.439741 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.439572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/55645792-0876-4299-acb7-665d6cabb951-cabundle-cert\") pod \"isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.439741 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.439596    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/55645792-0876-4299-acb7-665d6cabb951-proxy-tls\") pod \"isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.439741 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.439640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/55645792-0876-4299-acb7-665d6cabb951-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.439741 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.439696    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/55645792-0876-4299-acb7-665d6cabb951-kserve-provision-location\") pod \"isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.440193 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.440169    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/55645792-0876-4299-acb7-665d6cabb951-kserve-provision-location\") pod \"isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.440521 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.440494    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/55645792-0876-4299-acb7-665d6cabb951-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.440575 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.440532    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/55645792-0876-4299-acb7-665d6cabb951-cabundle-cert\") pod \"isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.442117 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.442101    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/55645792-0876-4299-acb7-665d6cabb951-proxy-tls\") pod \"isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.447921 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.447902    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fddg\" (UniqueName: \"kubernetes.io/projected/55645792-0876-4299-acb7-665d6cabb951-kube-api-access-8fddg\") pod \"isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.576915 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.576817    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:31.708524 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.708498    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"]
Aug 01 00:10:31.710880 ip-10-0-139-141 kubenswrapper[2581]: W0801 00:10:31.710845    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55645792_0876_4299_acb7_665d6cabb951.slice/crio-8f81eec3ddc52c738a03de6657e827d9bbc617c21ebbaece1390db4d0f9c76b1 WatchSource:0}: Error finding container 8f81eec3ddc52c738a03de6657e827d9bbc617c21ebbaece1390db4d0f9c76b1: Status 404 returned error can't find the container with id 8f81eec3ddc52c738a03de6657e827d9bbc617c21ebbaece1390db4d0f9c76b1
Aug 01 00:10:31.773056 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:31.773025    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" event={"ID":"55645792-0876-4299-acb7-665d6cabb951","Type":"ContainerStarted","Data":"8f81eec3ddc52c738a03de6657e827d9bbc617c21ebbaece1390db4d0f9c76b1"}
Aug 01 00:10:32.777894 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:32.777859    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" event={"ID":"55645792-0876-4299-acb7-665d6cabb951","Type":"ContainerStarted","Data":"d0e0d20ec4420ef50a0c57394714a0099ccd63f25becaf70dc27212fabddd98d"}
Aug 01 00:10:35.790344 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:35.790253    2581 generic.go:356] "Generic (PLEG): container finished" podID="55645792-0876-4299-acb7-665d6cabb951" containerID="d0e0d20ec4420ef50a0c57394714a0099ccd63f25becaf70dc27212fabddd98d" exitCode=0
Aug 01 00:10:35.790344 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:35.790328    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" event={"ID":"55645792-0876-4299-acb7-665d6cabb951","Type":"ContainerDied","Data":"d0e0d20ec4420ef50a0c57394714a0099ccd63f25becaf70dc27212fabddd98d"}
Aug 01 00:10:36.796864 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:36.796823    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" event={"ID":"55645792-0876-4299-acb7-665d6cabb951","Type":"ContainerStarted","Data":"5f1ecdd0fab7ac89c97b562f6e18627ced1d474bca64f7a91e522b021e4189e6"}
Aug 01 00:10:36.797463 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:36.796940    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" event={"ID":"55645792-0876-4299-acb7-665d6cabb951","Type":"ContainerStarted","Data":"e979dd1401526f635facae406ff200447034ee076ba4053fd885f0fad539dfe0"}
Aug 01 00:10:36.797463 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:36.797002    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" event={"ID":"55645792-0876-4299-acb7-665d6cabb951","Type":"ContainerStarted","Data":"2bc2e4c0959abc39a017ac9145a0a218fc9765894d33209ee4cd21f2cce39fff"}
Aug 01 00:10:36.797463 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:36.797417    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:36.797627 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:36.797476    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:36.797627 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:36.797518    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:36.799222 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:36.799185    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 01 00:10:36.799981 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:36.799936    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:10:36.821830 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:36.821776    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podStartSLOduration=2.206195515 podStartE2EDuration="5.821759828s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.615564313s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:10:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:10:36.819942528 +0000 UTC m=+1459.999399271" watchObservedRunningTime="2026-08-01 00:10:36.821759828 +0000 UTC m=+1460.001216572"
Aug 01 00:10:37.801430 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:37.801395    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 01 00:10:37.801876 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:37.801766    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:10:42.807204 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:42.807171    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:10:42.807864 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:42.807828    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 01 00:10:42.808372 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:42.808346    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:10:52.808124 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:52.808069    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 01 00:10:52.808664 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:10:52.808538    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:11:02.808131 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:11:02.808078    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 01 00:11:02.808656 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:11:02.808534    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:11:12.808136 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:11:12.808077    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 01 00:11:12.808649 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:11:12.808534    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:11:22.808643 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:11:22.808589    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 01 00:11:22.809261 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:11:22.809098    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:11:32.807745 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:11:32.807691    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 01 00:11:32.808277 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:11:32.808153    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:11:42.807977 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:11:42.807919    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 01 00:11:42.808510 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:11:42.808484    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:11:52.808663 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:11:52.808629    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:11:52.811063 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:11:52.809192    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:12:06.456286 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:06.456234    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"]
Aug 01 00:12:06.456761 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:06.456535    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kserve-container" containerID="cri-o://2bc2e4c0959abc39a017ac9145a0a218fc9765894d33209ee4cd21f2cce39fff" gracePeriod=30
Aug 01 00:12:06.456761 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:06.456554    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="agent" containerID="cri-o://5f1ecdd0fab7ac89c97b562f6e18627ced1d474bca64f7a91e522b021e4189e6" gracePeriod=30
Aug 01 00:12:06.456761 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:06.456611    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kube-rbac-proxy" containerID="cri-o://e979dd1401526f635facae406ff200447034ee076ba4053fd885f0fad539dfe0" gracePeriod=30
Aug 01 00:12:07.131182 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:07.131147    2581 generic.go:356] "Generic (PLEG): container finished" podID="55645792-0876-4299-acb7-665d6cabb951" containerID="e979dd1401526f635facae406ff200447034ee076ba4053fd885f0fad539dfe0" exitCode=2
Aug 01 00:12:07.131356 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:07.131220    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" event={"ID":"55645792-0876-4299-acb7-665d6cabb951","Type":"ContainerDied","Data":"e979dd1401526f635facae406ff200447034ee076ba4053fd885f0fad539dfe0"}
Aug 01 00:12:07.801977 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:07.801916    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.31:8643/healthz\": dial tcp 10.133.0.31:8643: connect: connection refused"
Aug 01 00:12:12.153102 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:12.153062    2581 generic.go:356] "Generic (PLEG): container finished" podID="55645792-0876-4299-acb7-665d6cabb951" containerID="2bc2e4c0959abc39a017ac9145a0a218fc9765894d33209ee4cd21f2cce39fff" exitCode=0
Aug 01 00:12:12.153492 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:12.153112    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" event={"ID":"55645792-0876-4299-acb7-665d6cabb951","Type":"ContainerDied","Data":"2bc2e4c0959abc39a017ac9145a0a218fc9765894d33209ee4cd21f2cce39fff"}
Aug 01 00:12:12.802416 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:12.802371    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.31:8643/healthz\": dial tcp 10.133.0.31:8643: connect: connection refused"
Aug 01 00:12:12.808140 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:12.808100    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:12:12.809344 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:12.809313    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 01 00:12:17.802066 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:17.802017    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.31:8643/healthz\": dial tcp 10.133.0.31:8643: connect: connection refused"
Aug 01 00:12:17.802535 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:17.802174    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:12:22.795778 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:22.795750    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kserve-container/0.log"
Aug 01 00:12:22.801673 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:22.801642    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.31:8643/healthz\": dial tcp 10.133.0.31:8643: connect: connection refused"
Aug 01 00:12:22.808813 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:22.808774    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:12:22.810101 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:22.810067    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 01 00:12:22.821750 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:22.821723    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kube-rbac-proxy/0.log"
Aug 01 00:12:22.832348 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:22.832320    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/agent/0.log"
Aug 01 00:12:22.843006 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:22.842952    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/storage-initializer/0.log"
Aug 01 00:12:23.625002 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:23.624971    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kserve-container/0.log"
Aug 01 00:12:23.636249 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:23.636220    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kube-rbac-proxy/0.log"
Aug 01 00:12:23.645910 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:23.645886    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/agent/0.log"
Aug 01 00:12:23.655847 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:23.655819    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/storage-initializer/0.log"
Aug 01 00:12:24.435022 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:24.434990    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kserve-container/0.log"
Aug 01 00:12:24.446236 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:24.446207    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kube-rbac-proxy/0.log"
Aug 01 00:12:24.455303 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:24.455272    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/agent/0.log"
Aug 01 00:12:24.464870 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:24.464818    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/storage-initializer/0.log"
Aug 01 00:12:25.222283 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:25.222253    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kserve-container/0.log"
Aug 01 00:12:25.233632 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:25.233599    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kube-rbac-proxy/0.log"
Aug 01 00:12:25.247689 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:25.243856    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/agent/0.log"
Aug 01 00:12:25.254233 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:25.254209    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/storage-initializer/0.log"
Aug 01 00:12:26.001917 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:26.001888    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kserve-container/0.log"
Aug 01 00:12:26.012749 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:26.012719    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kube-rbac-proxy/0.log"
Aug 01 00:12:26.021571 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:26.021548    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/agent/0.log"
Aug 01 00:12:26.032018 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:26.031995    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/storage-initializer/0.log"
Aug 01 00:12:26.768629 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:26.768595    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kserve-container/0.log"
Aug 01 00:12:26.780082 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:26.780054    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kube-rbac-proxy/0.log"
Aug 01 00:12:26.789784 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:26.789761    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/agent/0.log"
Aug 01 00:12:26.799188 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:26.799161    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/storage-initializer/0.log"
Aug 01 00:12:27.549395 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:27.549365    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kserve-container/0.log"
Aug 01 00:12:27.560363 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:27.560336    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kube-rbac-proxy/0.log"
Aug 01 00:12:27.570269 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:27.570242    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/agent/0.log"
Aug 01 00:12:27.580394 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:27.580354    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/storage-initializer/0.log"
Aug 01 00:12:27.801767 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:27.801676    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.31:8643/healthz\": dial tcp 10.133.0.31:8643: connect: connection refused"
Aug 01 00:12:28.344077 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:28.344045    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kserve-container/0.log"
Aug 01 00:12:28.352919 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:28.352890    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kube-rbac-proxy/0.log"
Aug 01 00:12:28.364045 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:28.364017    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/agent/0.log"
Aug 01 00:12:28.376067 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:28.376042    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/storage-initializer/0.log"
Aug 01 00:12:29.124802 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:29.124772    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kserve-container/0.log"
Aug 01 00:12:29.135967 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:29.135929    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kube-rbac-proxy/0.log"
Aug 01 00:12:29.145549 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:29.145525    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/agent/0.log"
Aug 01 00:12:29.155822 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:29.155799    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/storage-initializer/0.log"
Aug 01 00:12:29.898929 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:29.898899    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kserve-container/0.log"
Aug 01 00:12:29.910107 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:29.910082    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kube-rbac-proxy/0.log"
Aug 01 00:12:29.920401 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:29.920379    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/agent/0.log"
Aug 01 00:12:29.930937 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:29.930912    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/storage-initializer/0.log"
Aug 01 00:12:30.687499 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:30.687468    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kserve-container/0.log"
Aug 01 00:12:30.698077 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:30.698049    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kube-rbac-proxy/0.log"
Aug 01 00:12:30.707732 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:30.707706    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/agent/0.log"
Aug 01 00:12:30.717478 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:30.717455    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/storage-initializer/0.log"
Aug 01 00:12:31.449545 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:31.449519    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kserve-container/0.log"
Aug 01 00:12:31.460074 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:31.460047    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kube-rbac-proxy/0.log"
Aug 01 00:12:31.469130 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:31.469103    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/agent/0.log"
Aug 01 00:12:31.478834 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:31.478807    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/storage-initializer/0.log"
Aug 01 00:12:32.226436 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:32.226403    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kserve-container/0.log"
Aug 01 00:12:32.239630 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:32.239602    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kube-rbac-proxy/0.log"
Aug 01 00:12:32.251133 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:32.251107    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/agent/0.log"
Aug 01 00:12:32.264881 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:32.264856    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/storage-initializer/0.log"
Aug 01 00:12:32.802276 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:32.802221    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.31:8643/healthz\": dial tcp 10.133.0.31:8643: connect: connection refused"
Aug 01 00:12:32.809124 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:32.809085    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 01 00:12:32.809274 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:32.809227    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:12:32.809341 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:32.809309    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Aug 01 00:12:32.809431 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:32.809420    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:12:33.032132 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:33.032100    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kserve-container/0.log"
Aug 01 00:12:33.043225 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:33.043198    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/kube-rbac-proxy/0.log"
Aug 01 00:12:33.052985 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:33.052891    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/agent/0.log"
Aug 01 00:12:33.068843 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:33.068815    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh_55645792-0876-4299-acb7-665d6cabb951/storage-initializer/0.log"
Aug 01 00:12:36.596497 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:36.596469    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:12:36.693358 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:36.693319    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/55645792-0876-4299-acb7-665d6cabb951-proxy-tls\") pod \"55645792-0876-4299-acb7-665d6cabb951\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") "
Aug 01 00:12:36.693547 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:36.693381    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/55645792-0876-4299-acb7-665d6cabb951-cabundle-cert\") pod \"55645792-0876-4299-acb7-665d6cabb951\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") "
Aug 01 00:12:36.693547 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:36.693404    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/55645792-0876-4299-acb7-665d6cabb951-kube-rbac-proxy-sar-config\") pod \"55645792-0876-4299-acb7-665d6cabb951\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") "
Aug 01 00:12:36.693547 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:36.693433    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fddg\" (UniqueName: \"kubernetes.io/projected/55645792-0876-4299-acb7-665d6cabb951-kube-api-access-8fddg\") pod \"55645792-0876-4299-acb7-665d6cabb951\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") "
Aug 01 00:12:36.693547 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:36.693455    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/55645792-0876-4299-acb7-665d6cabb951-kserve-provision-location\") pod \"55645792-0876-4299-acb7-665d6cabb951\" (UID: \"55645792-0876-4299-acb7-665d6cabb951\") "
Aug 01 00:12:36.693796 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:36.693776    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55645792-0876-4299-acb7-665d6cabb951-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "55645792-0876-4299-acb7-665d6cabb951" (UID: "55645792-0876-4299-acb7-665d6cabb951"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 01 00:12:36.693850 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:36.693803    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55645792-0876-4299-acb7-665d6cabb951-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "55645792-0876-4299-acb7-665d6cabb951" (UID: "55645792-0876-4299-acb7-665d6cabb951"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:12:36.693850 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:36.693780    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55645792-0876-4299-acb7-665d6cabb951-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "55645792-0876-4299-acb7-665d6cabb951" (UID: "55645792-0876-4299-acb7-665d6cabb951"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:12:36.695545 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:36.695519    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55645792-0876-4299-acb7-665d6cabb951-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "55645792-0876-4299-acb7-665d6cabb951" (UID: "55645792-0876-4299-acb7-665d6cabb951"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 01 00:12:36.695650 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:36.695563    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55645792-0876-4299-acb7-665d6cabb951-kube-api-access-8fddg" (OuterVolumeSpecName: "kube-api-access-8fddg") pod "55645792-0876-4299-acb7-665d6cabb951" (UID: "55645792-0876-4299-acb7-665d6cabb951"). InnerVolumeSpecName "kube-api-access-8fddg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:12:36.794318 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:36.794221    2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/55645792-0876-4299-acb7-665d6cabb951-proxy-tls\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:12:36.794318 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:36.794265    2581 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/55645792-0876-4299-acb7-665d6cabb951-cabundle-cert\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:12:36.794318 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:36.794275    2581 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/55645792-0876-4299-acb7-665d6cabb951-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:12:36.794318 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:36.794284    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fddg\" (UniqueName: \"kubernetes.io/projected/55645792-0876-4299-acb7-665d6cabb951-kube-api-access-8fddg\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:12:36.794318 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:36.794295    2581 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/55645792-0876-4299-acb7-665d6cabb951-kserve-provision-location\") on node \"ip-10-0-139-141.ec2.internal\" DevicePath \"\""
Aug 01 00:12:37.243789 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.243750    2581 generic.go:356] "Generic (PLEG): container finished" podID="55645792-0876-4299-acb7-665d6cabb951" containerID="5f1ecdd0fab7ac89c97b562f6e18627ced1d474bca64f7a91e522b021e4189e6" exitCode=0
Aug 01 00:12:37.243982 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.243825    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"
Aug 01 00:12:37.243982 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.243832    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" event={"ID":"55645792-0876-4299-acb7-665d6cabb951","Type":"ContainerDied","Data":"5f1ecdd0fab7ac89c97b562f6e18627ced1d474bca64f7a91e522b021e4189e6"}
Aug 01 00:12:37.243982 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.243901    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh" event={"ID":"55645792-0876-4299-acb7-665d6cabb951","Type":"ContainerDied","Data":"8f81eec3ddc52c738a03de6657e827d9bbc617c21ebbaece1390db4d0f9c76b1"}
Aug 01 00:12:37.243982 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.243933    2581 scope.go:117] "RemoveContainer" containerID="5f1ecdd0fab7ac89c97b562f6e18627ced1d474bca64f7a91e522b021e4189e6"
Aug 01 00:12:37.252215 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.252193    2581 scope.go:117] "RemoveContainer" containerID="e979dd1401526f635facae406ff200447034ee076ba4053fd885f0fad539dfe0"
Aug 01 00:12:37.259647 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.259624    2581 scope.go:117] "RemoveContainer" containerID="2bc2e4c0959abc39a017ac9145a0a218fc9765894d33209ee4cd21f2cce39fff"
Aug 01 00:12:37.265465 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.265438    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"]
Aug 01 00:12:37.267165 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.267136    2581 scope.go:117] "RemoveContainer" containerID="d0e0d20ec4420ef50a0c57394714a0099ccd63f25becaf70dc27212fabddd98d"
Aug 01 00:12:37.269215 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.269193    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-fc5d477ff-k7bnh"]
Aug 01 00:12:37.274710 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.274692    2581 scope.go:117] "RemoveContainer" containerID="5f1ecdd0fab7ac89c97b562f6e18627ced1d474bca64f7a91e522b021e4189e6"
Aug 01 00:12:37.275013 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:12:37.274985    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f1ecdd0fab7ac89c97b562f6e18627ced1d474bca64f7a91e522b021e4189e6\": container with ID starting with 5f1ecdd0fab7ac89c97b562f6e18627ced1d474bca64f7a91e522b021e4189e6 not found: ID does not exist" containerID="5f1ecdd0fab7ac89c97b562f6e18627ced1d474bca64f7a91e522b021e4189e6"
Aug 01 00:12:37.275076 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.275028    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f1ecdd0fab7ac89c97b562f6e18627ced1d474bca64f7a91e522b021e4189e6"} err="failed to get container status \"5f1ecdd0fab7ac89c97b562f6e18627ced1d474bca64f7a91e522b021e4189e6\": rpc error: code = NotFound desc = could not find container \"5f1ecdd0fab7ac89c97b562f6e18627ced1d474bca64f7a91e522b021e4189e6\": container with ID starting with 5f1ecdd0fab7ac89c97b562f6e18627ced1d474bca64f7a91e522b021e4189e6 not found: ID does not exist"
Aug 01 00:12:37.275076 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.275055    2581 scope.go:117] "RemoveContainer" containerID="e979dd1401526f635facae406ff200447034ee076ba4053fd885f0fad539dfe0"
Aug 01 00:12:37.275342 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:12:37.275323    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e979dd1401526f635facae406ff200447034ee076ba4053fd885f0fad539dfe0\": container with ID starting with e979dd1401526f635facae406ff200447034ee076ba4053fd885f0fad539dfe0 not found: ID does not exist" containerID="e979dd1401526f635facae406ff200447034ee076ba4053fd885f0fad539dfe0"
Aug 01 00:12:37.275396 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.275350    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e979dd1401526f635facae406ff200447034ee076ba4053fd885f0fad539dfe0"} err="failed to get container status \"e979dd1401526f635facae406ff200447034ee076ba4053fd885f0fad539dfe0\": rpc error: code = NotFound desc = could not find container \"e979dd1401526f635facae406ff200447034ee076ba4053fd885f0fad539dfe0\": container with ID starting with e979dd1401526f635facae406ff200447034ee076ba4053fd885f0fad539dfe0 not found: ID does not exist"
Aug 01 00:12:37.275396 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.275371    2581 scope.go:117] "RemoveContainer" containerID="2bc2e4c0959abc39a017ac9145a0a218fc9765894d33209ee4cd21f2cce39fff"
Aug 01 00:12:37.275612 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:12:37.275594    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bc2e4c0959abc39a017ac9145a0a218fc9765894d33209ee4cd21f2cce39fff\": container with ID starting with 2bc2e4c0959abc39a017ac9145a0a218fc9765894d33209ee4cd21f2cce39fff not found: ID does not exist" containerID="2bc2e4c0959abc39a017ac9145a0a218fc9765894d33209ee4cd21f2cce39fff"
Aug 01 00:12:37.275658 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.275617    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bc2e4c0959abc39a017ac9145a0a218fc9765894d33209ee4cd21f2cce39fff"} err="failed to get container status \"2bc2e4c0959abc39a017ac9145a0a218fc9765894d33209ee4cd21f2cce39fff\": rpc error: code = NotFound desc = could not find container \"2bc2e4c0959abc39a017ac9145a0a218fc9765894d33209ee4cd21f2cce39fff\": container with ID starting with 2bc2e4c0959abc39a017ac9145a0a218fc9765894d33209ee4cd21f2cce39fff not found: ID does not exist"
Aug 01 00:12:37.275658 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.275632    2581 scope.go:117] "RemoveContainer" containerID="d0e0d20ec4420ef50a0c57394714a0099ccd63f25becaf70dc27212fabddd98d"
Aug 01 00:12:37.275859 ip-10-0-139-141 kubenswrapper[2581]: E0801 00:12:37.275844    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0e0d20ec4420ef50a0c57394714a0099ccd63f25becaf70dc27212fabddd98d\": container with ID starting with d0e0d20ec4420ef50a0c57394714a0099ccd63f25becaf70dc27212fabddd98d not found: ID does not exist" containerID="d0e0d20ec4420ef50a0c57394714a0099ccd63f25becaf70dc27212fabddd98d"
Aug 01 00:12:37.275900 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.275861    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0e0d20ec4420ef50a0c57394714a0099ccd63f25becaf70dc27212fabddd98d"} err="failed to get container status \"d0e0d20ec4420ef50a0c57394714a0099ccd63f25becaf70dc27212fabddd98d\": rpc error: code = NotFound desc = could not find container \"d0e0d20ec4420ef50a0c57394714a0099ccd63f25becaf70dc27212fabddd98d\": container with ID starting with d0e0d20ec4420ef50a0c57394714a0099ccd63f25becaf70dc27212fabddd98d not found: ID does not exist"
Aug 01 00:12:37.387163 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:37.387129    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55645792-0876-4299-acb7-665d6cabb951" path="/var/lib/kubelet/pods/55645792-0876-4299-acb7-665d6cabb951/volumes"
Aug 01 00:12:41.824334 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:41.824298    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4k4dl_6c1ae62a-0b92-4205-a4bb-5d266930b743/global-pull-secret-syncer/0.log"
Aug 01 00:12:41.997253 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:41.997218    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zqrmw_bb83a5d0-9c9a-4d84-8562-560971cad828/konnectivity-agent/0.log"
Aug 01 00:12:42.061695 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:42.061658    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-141.ec2.internal_109099d262acad3a59990c8222269413/haproxy/0.log"
Aug 01 00:12:46.498436 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:46.498404    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-727b8_7e5618be-8358-4dba-aa6e-3078a1f8c05b/node-exporter/0.log"
Aug 01 00:12:46.518403 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:46.518378    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-727b8_7e5618be-8358-4dba-aa6e-3078a1f8c05b/kube-rbac-proxy/0.log"
Aug 01 00:12:46.537886 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:46.537857    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-727b8_7e5618be-8358-4dba-aa6e-3078a1f8c05b/init-textfile/0.log"
Aug 01 00:12:49.381968 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:49.381922    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-645997df77-hps8r_2a9f0928-d041-4f5e-9539-8281ea50ecb3/console/0.log"
Aug 01 00:12:49.997502 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:49.997462    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"]
Aug 01 00:12:49.997729 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:49.997717    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="storage-initializer"
Aug 01 00:12:49.997774 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:49.997731    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="storage-initializer"
Aug 01 00:12:49.997774 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:49.997740    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kube-rbac-proxy"
Aug 01 00:12:49.997774 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:49.997746    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kube-rbac-proxy"
Aug 01 00:12:49.997774 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:49.997757    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="agent"
Aug 01 00:12:49.997774 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:49.997763    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="agent"
Aug 01 00:12:49.997774 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:49.997771    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kserve-container"
Aug 01 00:12:49.997774 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:49.997776    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kserve-container"
Aug 01 00:12:49.998022 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:49.997815    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kserve-container"
Aug 01 00:12:49.998022 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:49.997822    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="agent"
Aug 01 00:12:49.998022 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:49.997828    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="55645792-0876-4299-acb7-665d6cabb951" containerName="kube-rbac-proxy"
Aug 01 00:12:50.000507 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.000488    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.002917 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.002895    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xgqkp\"/\"openshift-service-ca.crt\""
Aug 01 00:12:50.004068 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.004047    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xgqkp\"/\"default-dockercfg-kblfc\""
Aug 01 00:12:50.004155 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.004057    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xgqkp\"/\"kube-root-ca.crt\""
Aug 01 00:12:50.010023 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.009999    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"]
Aug 01 00:12:50.096363 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.096330    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87ce38d1-5ad5-4c8f-ad1c-d5586e29e150-lib-modules\") pod \"perf-node-gather-daemonset-w8ltb\" (UID: \"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.096583 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.096375    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/87ce38d1-5ad5-4c8f-ad1c-d5586e29e150-proc\") pod \"perf-node-gather-daemonset-w8ltb\" (UID: \"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.096583 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.096511    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fccjq\" (UniqueName: \"kubernetes.io/projected/87ce38d1-5ad5-4c8f-ad1c-d5586e29e150-kube-api-access-fccjq\") pod \"perf-node-gather-daemonset-w8ltb\" (UID: \"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.096583 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.096548    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87ce38d1-5ad5-4c8f-ad1c-d5586e29e150-sys\") pod \"perf-node-gather-daemonset-w8ltb\" (UID: \"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.096725 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.096602    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/87ce38d1-5ad5-4c8f-ad1c-d5586e29e150-podres\") pod \"perf-node-gather-daemonset-w8ltb\" (UID: \"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.197815 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.197772    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/87ce38d1-5ad5-4c8f-ad1c-d5586e29e150-proc\") pod \"perf-node-gather-daemonset-w8ltb\" (UID: \"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.198050 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.197832    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fccjq\" (UniqueName: \"kubernetes.io/projected/87ce38d1-5ad5-4c8f-ad1c-d5586e29e150-kube-api-access-fccjq\") pod \"perf-node-gather-daemonset-w8ltb\" (UID: \"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.198050 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.197857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87ce38d1-5ad5-4c8f-ad1c-d5586e29e150-sys\") pod \"perf-node-gather-daemonset-w8ltb\" (UID: \"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.198050 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.197891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/87ce38d1-5ad5-4c8f-ad1c-d5586e29e150-podres\") pod \"perf-node-gather-daemonset-w8ltb\" (UID: \"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.198050 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.197926    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87ce38d1-5ad5-4c8f-ad1c-d5586e29e150-sys\") pod \"perf-node-gather-daemonset-w8ltb\" (UID: \"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.198050 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.197921    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/87ce38d1-5ad5-4c8f-ad1c-d5586e29e150-proc\") pod \"perf-node-gather-daemonset-w8ltb\" (UID: \"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.198050 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.197933    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87ce38d1-5ad5-4c8f-ad1c-d5586e29e150-lib-modules\") pod \"perf-node-gather-daemonset-w8ltb\" (UID: \"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.198293 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.198053    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/87ce38d1-5ad5-4c8f-ad1c-d5586e29e150-podres\") pod \"perf-node-gather-daemonset-w8ltb\" (UID: \"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.198293 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.198071    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87ce38d1-5ad5-4c8f-ad1c-d5586e29e150-lib-modules\") pod \"perf-node-gather-daemonset-w8ltb\" (UID: \"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.208796 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.208761    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fccjq\" (UniqueName: \"kubernetes.io/projected/87ce38d1-5ad5-4c8f-ad1c-d5586e29e150-kube-api-access-fccjq\") pod \"perf-node-gather-daemonset-w8ltb\" (UID: \"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150\") " pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.311030 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.310927    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:50.444332 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.444294    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"]
Aug 01 00:12:50.447437 ip-10-0-139-141 kubenswrapper[2581]: W0801 00:12:50.447409    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod87ce38d1_5ad5_4c8f_ad1c_d5586e29e150.slice/crio-c20f768a31e834e1d69bceb981e767606cbf85764add744a228640b687c75915 WatchSource:0}: Error finding container c20f768a31e834e1d69bceb981e767606cbf85764add744a228640b687c75915: Status 404 returned error can't find the container with id c20f768a31e834e1d69bceb981e767606cbf85764add744a228640b687c75915
Aug 01 00:12:50.907472 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.907443    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x8242_4c461a2c-8de2-4813-9cfd-45366645a56e/dns/0.log"
Aug 01 00:12:50.927251 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.927223    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x8242_4c461a2c-8de2-4813-9cfd-45366645a56e/kube-rbac-proxy/0.log"
Aug 01 00:12:50.969768 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:50.969736    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-twdx5_f85fb808-2a50-499e-a3a8-3aa1c33f69a2/dns-node-resolver/0.log"
Aug 01 00:12:51.298576 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:51.298472    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb" event={"ID":"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150","Type":"ContainerStarted","Data":"03f7ee9fe9d362cb328f84fecf53ab1695bc8a9fc20e2f917139a32436236898"}
Aug 01 00:12:51.298576 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:51.298529    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb" event={"ID":"87ce38d1-5ad5-4c8f-ad1c-d5586e29e150","Type":"ContainerStarted","Data":"c20f768a31e834e1d69bceb981e767606cbf85764add744a228640b687c75915"}
Aug 01 00:12:51.298810 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:51.298582    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:12:51.314468 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:51.314413    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb" podStartSLOduration=2.314396975 podStartE2EDuration="2.314396975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-01 00:12:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:12:51.313279997 +0000 UTC m=+1594.492736740" watchObservedRunningTime="2026-08-01 00:12:51.314396975 +0000 UTC m=+1594.493853762"
Aug 01 00:12:51.424402 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:51.424368    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29759040-jszhz_eca07b31-b4fa-44f7-9c31-db23e16be4c5/image-pruner/0.log"
Aug 01 00:12:51.497319 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:51.497289    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d25h7_98dd5ea5-e96e-406a-956c-200a1e0314d1/node-ca/0.log"
Aug 01 00:12:52.547587 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:52.547553    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-67cs9_72d94379-6a47-4f1a-af3b-25e254074711/serve-healthcheck-canary/0.log"
Aug 01 00:12:52.989410 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:52.989379    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dcbx4_4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f/kube-rbac-proxy/0.log"
Aug 01 00:12:53.006750 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:53.006716    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dcbx4_4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f/exporter/0.log"
Aug 01 00:12:53.025223 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:53.025197    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dcbx4_4c3b0afb-4808-4f97-9ac3-c0a7a13d4b2f/extractor/0.log"
Aug 01 00:12:55.296190 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:55.296157    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-654c557856-kcgr2_bbe6029c-9520-421d-b0ca-d6b283c65a20/manager/0.log"
Aug 01 00:12:55.315787 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:55.315758    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-7b954ffc95-xvsbm_8d19a962-96c7-406c-8a33-42b17e748ab4/server/0.log"
Aug 01 00:12:57.311442 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:12:57.311412    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xgqkp/perf-node-gather-daemonset-w8ltb"
Aug 01 00:13:00.661910 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:00.661877    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q6bv6_9da928a7-a1b0-4b64-a8a0-0e007b337734/kube-multus-additional-cni-plugins/0.log"
Aug 01 00:13:00.685738 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:00.685654    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q6bv6_9da928a7-a1b0-4b64-a8a0-0e007b337734/egress-router-binary-copy/0.log"
Aug 01 00:13:00.706282 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:00.706255    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q6bv6_9da928a7-a1b0-4b64-a8a0-0e007b337734/cni-plugins/0.log"
Aug 01 00:13:00.725917 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:00.725893    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q6bv6_9da928a7-a1b0-4b64-a8a0-0e007b337734/bond-cni-plugin/0.log"
Aug 01 00:13:00.749652 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:00.749621    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q6bv6_9da928a7-a1b0-4b64-a8a0-0e007b337734/routeoverride-cni/0.log"
Aug 01 00:13:00.769880 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:00.769854    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q6bv6_9da928a7-a1b0-4b64-a8a0-0e007b337734/whereabouts-cni-bincopy/0.log"
Aug 01 00:13:00.790921 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:00.790896    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q6bv6_9da928a7-a1b0-4b64-a8a0-0e007b337734/whereabouts-cni/0.log"
Aug 01 00:13:00.980903 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:00.980818    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-hftd5_5dd3564c-d0e5-444f-9417-4d23ea2951e1/kube-multus/0.log"
Aug 01 00:13:01.096147 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:01.096112    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tn7kj_75c8e3ed-6587-4ef9-b3bb-562bf74fec5b/network-metrics-daemon/0.log"
Aug 01 00:13:01.114212 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:01.114181    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tn7kj_75c8e3ed-6587-4ef9-b3bb-562bf74fec5b/kube-rbac-proxy/0.log"
Aug 01 00:13:01.866614 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:01.866585    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5sqvw_4869dda6-8fe5-4c60-abc7-f16ce429b2d4/ovn-controller/0.log"
Aug 01 00:13:01.883252 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:01.883225    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5sqvw_4869dda6-8fe5-4c60-abc7-f16ce429b2d4/ovn-acl-logging/0.log"
Aug 01 00:13:01.903439 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:01.903406    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5sqvw_4869dda6-8fe5-4c60-abc7-f16ce429b2d4/kube-rbac-proxy-node/0.log"
Aug 01 00:13:01.922374 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:01.922345    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5sqvw_4869dda6-8fe5-4c60-abc7-f16ce429b2d4/kube-rbac-proxy-ovn-metrics/0.log"
Aug 01 00:13:01.940808 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:01.940777    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5sqvw_4869dda6-8fe5-4c60-abc7-f16ce429b2d4/northd/0.log"
Aug 01 00:13:01.959456 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:01.959432    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5sqvw_4869dda6-8fe5-4c60-abc7-f16ce429b2d4/nbdb/0.log"
Aug 01 00:13:01.978463 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:01.978429    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5sqvw_4869dda6-8fe5-4c60-abc7-f16ce429b2d4/sbdb/0.log"
Aug 01 00:13:02.014684 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:02.014647    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5sqvw_4869dda6-8fe5-4c60-abc7-f16ce429b2d4/ovnkube-controller/0.log"
Aug 01 00:13:03.578621 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:03.578591    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-frb4r_1cc3f661-c981-4802-964e-632fab8107c6/network-check-target-container/0.log"
Aug 01 00:13:04.545762 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:04.545736    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-c8qbc_6c600206-7804-41f1-ae49-de6c40a4d2e0/iptables-alerter/0.log"
Aug 01 00:13:05.197064 ip-10-0-139-141 kubenswrapper[2581]: I0801 00:13:05.197037    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-dpmjk_d8685c5a-294f-4e78-9504-085286bd468a/tuned/0.log"