Jul 29 10:27:01.487324 ip-10-0-133-233 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 29 10:27:01.487339 ip-10-0-133-233 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 29 10:27:01.487366 ip-10-0-133-233 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 29 10:27:01.487684 ip-10-0-133-233 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 29 10:27:11.674896 ip-10-0-133-233 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 29 10:27:11.674914 ip-10-0-133-233 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot f095bd1de2c341659a55c0f695a1076c --
Jul 29 10:28:47.632722 ip-10-0-133-233 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 10:28:48.054334 ip-10-0-133-233 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 10:28:48.054334 ip-10-0-133-233 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 10:28:48.054334 ip-10-0-133-233 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 10:28:48.054334 ip-10-0-133-233 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 10:28:48.054334 ip-10-0-133-233 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 10:28:48.057316 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.057233    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 10:28:48.065126 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065100    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 10:28:48.065126 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065120    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 10:28:48.065126 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065124    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 10:28:48.065126 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065127    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 10:28:48.065126 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065130    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 10:28:48.065126 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065133    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 10:28:48.065126 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065137    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065140    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065143    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065146    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065148    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065152    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065154    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065157    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065160    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065162    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065165    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065168    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065171    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065174    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065177    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065180    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065182    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065193    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065196    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 10:28:48.065363 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065199    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065201    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065204    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065207    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065209    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065212    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065214    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065217    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065220    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065223    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065226    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065228    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065231    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065233    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065236    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065239    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065242    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065245    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065248    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065251    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065253    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 10:28:48.065868 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065256    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065258    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065262    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065264    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065267    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065272    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065276    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065279    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065282    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065286    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065290    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065294    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065297    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065300    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065303    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065306    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065309    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065312    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065315    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 10:28:48.066379 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065318    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065321    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065323    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065326    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065329    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065333    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065337    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065341    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065344    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065346    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065349    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065352    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065354    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065357    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065360    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065363    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065365    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065368    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065373    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 10:28:48.066854 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065376    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065378    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065381    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065383    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065386    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065388    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065392    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065394    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065397    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065399    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065401    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065404    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065407    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065410    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.065412    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066130    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066138    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066141    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066143    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066146    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 10:28:48.067323 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066149    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066152    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066154    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066157    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066160    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066162    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066165    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066168    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066170    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066173    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066175    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066178    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066181    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066184    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066187    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066192    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066196    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066199    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066202    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 10:28:48.067822 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066205    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066207    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066210    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066213    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066216    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066218    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066221    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066223    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066226    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066228    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066230    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066233    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066235    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066239    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066241    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066244    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066246    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066249    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066252    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 10:28:48.068334 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066254    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066257    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066260    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066262    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066265    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066268    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066271    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066274    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066278    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066280    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066283    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066285    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066288    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066292    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066295    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066298    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066301    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066304    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066307    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 10:28:48.068803 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066310    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066313    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066315    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066318    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066321    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066323    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066326    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066329    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066331    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066334    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066336    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066339    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066341    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066344    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066346    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066349    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066351    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066354    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066358    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066360    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 10:28:48.069270 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066362    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066366    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066369    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066371    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066375    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066380    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066383    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066386    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066388    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066391    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066394    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066396    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066399    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066402    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066404    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066407    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066410    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066789    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066794    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066797    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 10:28:48.069789 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066800    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066803    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066805    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066808    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066811    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066816    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066819    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066823    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066826    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066829    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066831    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066834    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066836    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066840    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066843    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066846    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066848    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066851    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066854    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 10:28:48.070273 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066856    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066860    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066863    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066865    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066869    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066871    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066874    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066876    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066879    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066882    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066884    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066887    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066890    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066894    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066897    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066900    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066903    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066906    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066908    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 10:28:48.070758 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066911    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066914    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066916    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066919    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066921    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066924    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066926    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066929    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066932    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066934    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066937    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066939    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066942    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066945    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066947    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066950    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066953    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066956    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066959    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066961    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 10:28:48.071213 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066964    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066966    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066970    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066972    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066975    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066978    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066980    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066982    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066986    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066988    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066991    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066993    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066996    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.066998    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067001    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067003    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067006    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067008    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067011    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067014    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 10:28:48.071709 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067016    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067019    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067021    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067025    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067027    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067030    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067033    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067036    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067039    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067042    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067044    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067047    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067049    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067052    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067055    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067057    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067060    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067062    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067133    2564 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067144    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 10:28:48.072198 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067153    2564 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067157    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067161    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067165    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067169    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067173    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067176    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067179    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067183    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067186    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067189    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067192    2564 flags.go:64] FLAG: --cgroup-root=""
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067195    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067198    2564 flags.go:64] FLAG: --client-ca-file=""
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067201    2564 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067204    2564 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067209    2564 flags.go:64] FLAG: --cluster-domain=""
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067212    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067217    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067221    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067224    2564 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067228    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067231    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 10:28:48.072699 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067234    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067237    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067240    2564 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067243    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067246    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067249    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067252    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067256    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067259    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067262    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067265    2564 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067268    2564 flags.go:64] FLAG: --enable-server="true"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067271    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067276    2564 flags.go:64] FLAG: --event-burst="100"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067279    2564 flags.go:64] FLAG: --event-qps="50"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067281    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067284    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067287    2564 flags.go:64] FLAG: --eviction-hard=""
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067291    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067294    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067297    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067300    2564 flags.go:64] FLAG: --eviction-soft=""
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067303    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067306    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067309    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 10:28:48.073248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067312    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067315    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067320    2564 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067323    2564 flags.go:64] FLAG: --feature-gates=""
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067328    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067331    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067334    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067337    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067340    2564 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067343    2564 flags.go:64] FLAG: --help="false"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067346    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067349    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067352    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067354    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067358    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067361    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067364    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067367    2564 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067370    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067373    2564 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067375    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067378    2564 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067381    2564 flags.go:64] FLAG: --kube-reserved=""
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067384    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 10:28:48.073917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067387    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067390    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067392    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067395    2564 flags.go:64] FLAG: --lock-file=""
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067398    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067400    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067403    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067408    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067412    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067415    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067419    2564 flags.go:64] FLAG: --logging-format="text"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067423    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067427    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067430    2564 flags.go:64] FLAG: --manifest-url=""
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067433    2564 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067437    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067440    2564 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067459    2564 flags.go:64] FLAG: --max-pods="110"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067463    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067466    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067469    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067473    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067475    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067478    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067482    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 10:28:48.074556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067490    2564 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067493    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067496    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067499    2564 flags.go:64] FLAG: --pod-cidr=""
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067501    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067507    2564 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067510    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067512    2564 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067515    2564 flags.go:64] FLAG: --port="10250"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067518    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067522    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b49d4013f4b1f651"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067525    2564 flags.go:64] FLAG: --qos-reserved=""
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067528    2564 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067531    2564 flags.go:64] FLAG: --register-node="true"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067534    2564 flags.go:64] FLAG: --register-with-taints=""
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067537    2564 flags.go:64] FLAG: --registry-burst="10"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067541    2564 flags.go:64] FLAG: --registry-qps="5"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067546    2564 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067548    2564 flags.go:64] FLAG: --reserved-memory=""
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067553    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067556    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067559    2564 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067562    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067565    2564 flags.go:64] FLAG: --runonce="false"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067568    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 10:28:48.075161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067571    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067574    2564 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067577    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067580    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067583    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067586    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067589    2564 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067592    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067595    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067598    2564 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067600    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067603    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067606    2564 flags.go:64] FLAG: --system-cgroups=""
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067609    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067614    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067617    2564 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067620    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067624    2564 flags.go:64] FLAG: --tls-min-version=""
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067627    2564 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067629    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067632    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067635    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067638    2564 flags.go:64] FLAG: --v="2"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067642    2564 flags.go:64] FLAG: --version="false"
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067647    2564 flags.go:64] FLAG: --vmodule=""
Jul 29 10:28:48.075793 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067651    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.067656    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067773    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067777    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067781    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067791    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067795    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067797    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067800    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067804    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067806    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067809    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067812    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067814    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067818    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067822    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067825    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067828    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067831    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067834    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 10:28:48.076395 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067837    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067840    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067842    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067845    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067848    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067850    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067853    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067855    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067858    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067860    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067863    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067868    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067871    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067875    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067878    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067881    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067883    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067886    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067888    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067891    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 10:28:48.076893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067893    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067896    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067898    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067901    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067904    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067906    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067909    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067912    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067914    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067917    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067919    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067922    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067924    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067927    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067929    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067932    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067935    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067938    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067942    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 10:28:48.077429 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067945    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067948    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067950    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067953    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067956    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067959    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067970    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067973    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067975    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067978    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067981    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067983    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067986    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067988    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067991    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067993    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067996    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.067998    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068001    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068004    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 10:28:48.077904 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068006    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068009    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068011    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068014    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068017    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068019    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068021    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068024    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068026    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068029    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068032    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068034    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068037    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068040    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068044    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068047    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068051    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068059    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068064    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 10:28:48.078400 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068072    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 10:28:48.078867 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068075    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 10:28:48.078867 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.068078    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 10:28:48.078867 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.068914    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 10:28:48.078867 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.075621    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 10:28:48.078867 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.075636    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 10:28:48.078867 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.076775    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 10:28:48.078867 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.076786    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 10:28:48.078867 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076862    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 10:28:48.078867 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076867    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 10:28:48.078867 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076870    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 10:28:48.078867 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076874    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 10:28:48.078867 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076877    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 10:28:48.078867 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076880    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 10:28:48.078867 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076883    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 10:28:48.078867 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076885    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076888    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076891    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076894    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076896    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076899    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076902    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076905    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076907    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076910    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076913    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076916    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076918    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076921    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076924    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076926    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076929    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076932    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076935    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076938    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 10:28:48.079233 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076940    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076943    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076946    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076949    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076952    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076954    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076957    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076960    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076963    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076965    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076968    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076970    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076973    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076976    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076978    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076981    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076984    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076986    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076989    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076992    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 10:28:48.079725 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076994    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.076997    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077000    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077002    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077005    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077007    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077010    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077012    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077015    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077017    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077026    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077029    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077031    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077036    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077040    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077044    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077047    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077050    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077053    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 10:28:48.080267 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077055    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077058    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077061    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077064    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077066    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077069    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077071    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077074    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077077    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077079    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077082    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077085    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077087    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077089    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077092    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077095    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077097    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077100    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077102    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077105    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 10:28:48.080828 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077108    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 10:28:48.081321 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077110    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 10:28:48.081321 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077113    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 10:28:48.081321 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077117    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 10:28:48.081321 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077125    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 10:28:48.081321 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077129    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 10:28:48.081321 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077131    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 10:28:48.081321 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077135    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 10:28:48.081321 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077138    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 10:28:48.081321 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077142    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 10:28:48.081321 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077146    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 10:28:48.081321 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077148    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 10:28:48.081321 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077151    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 10:28:48.081321 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.077156    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 10:28:48.081321 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077295    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077300    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077303    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077306    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077309    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077318    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077322    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077326    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077330    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077333    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077336    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077339    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077341    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077344    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077346    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077349    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077352    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077354    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077357    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 10:28:48.081720 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077359    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077362    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077364    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077367    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077376    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077379    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077383    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077386    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077389    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077391    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077394    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077397    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077399    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077402    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077404    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077406    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077409    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077411    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077414    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077416    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 10:28:48.082188 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077419    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077421    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077424    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077426    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077429    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077431    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077434    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077436    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077439    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077454    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077459    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077462    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077464    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077466    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077469    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077472    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077474    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077483    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077486    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 10:28:48.082691 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077489    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077492    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077495    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077497    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077500    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077502    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077505    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077507    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077510    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077513    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077515    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077518    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077521    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077523    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077526    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077528    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077531    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077533    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077536    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077538    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 10:28:48.083155 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077541    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077543    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077545    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077548    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077551    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077555    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077558    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077561    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077564    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077567    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077575    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077584    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077587    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077590    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077592    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077595    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077598    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077601    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077603    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 10:28:48.083655 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077606    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 10:28:48.084127 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:48.077608    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 10:28:48.084127 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.077613    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 10:28:48.084127 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.077717    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 10:28:48.084127 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.079752    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 10:28:48.084127 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.081349    2564 server.go:1019] "Starting client certificate rotation"
Jul 29 10:28:48.084127 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.081476    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 10:28:48.084127 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.081509    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 10:28:48.084341 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.084184    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 10:28:48.085852 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.085838    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 10:28:48.094196 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.094177    2564 log.go:25] "Validated CRI v1 runtime API"
Jul 29 10:28:48.097164 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.097146    2564 log.go:25] "Validated CRI v1 image API"
Jul 29 10:28:48.097990 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.097972    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 10:28:48.101115 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.101082    2564 fs.go:135] Filesystem UUIDs: map[05d3c3e2-cd76-4ccc-9858-274c24f1bc0d:/dev/nvme0n1p4 7132308c-2caf-45d9-ab71-4e27b0b466d0:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 29 10:28:48.101185 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.101114    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 10:28:48.106919 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.106903    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 10:28:48.107620 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.107510    2564 manager.go:216] Machine: {Timestamp:2026-07-29 10:28:48.106073994 +0000 UTC m=+0.356784692 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106465 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26fd24dc7202ae3f8b25c989c25e3b SystemUUID:ec26fd24-dc72-02ae-3f8b-25c989c25e3b BootID:f095bd1d-e2c3-4165-9a55-c0f695a1076c Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0d:09:55:64:6d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0d:09:55:64:6d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:27:a9:39:04:d4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 10:28:48.107680 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.107620    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 10:28:48.107709 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.107701    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 10:28:48.110523 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.110500    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 10:28:48.110712 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.110526    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-233.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 10:28:48.111842 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.111820    2564 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 10:28:48.111875 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.111846    2564 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 10:28:48.111875 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.111853    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 10:28:48.112624 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.112613    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 10:28:48.113425 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.113416    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 10:28:48.115038 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.115027    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 10:28:48.115155 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.115146    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 10:28:48.115187 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.115177    2564 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 10:28:48.115217 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.115202    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 10:28:48.115245 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.115218    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 10:28:48.115245 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.115234    2564 kubelet.go:398] "Adding apiserver pod source"
Jul 29 10:28:48.115245 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.115244    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 10:28:48.115679 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.115665    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 10:28:48.115742 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.115684    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 10:28:48.119938 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.119913    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 10:28:48.122066 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.122053    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 10:28:48.122124 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.122115    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 10:28:48.123617 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.123605    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 10:28:48.123658 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.123621    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 10:28:48.123658 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.123627    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 10:28:48.123658 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.123633    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 10:28:48.123658 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.123642    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 10:28:48.123658 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.123647    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 10:28:48.123658 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.123657    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 10:28:48.123824 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.123662    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 10:28:48.123824 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.123669    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 10:28:48.123824 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.123675    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 10:28:48.123824 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.123684    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 10:28:48.123824 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.123689    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 10:28:48.124691 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.124668    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 10:28:48.124722 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.124700    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 10:28:48.124782 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.124761    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 10:28:48.124848 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.124830    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-233.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 10:28:48.127654 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.127638    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-884pg"
Jul 29 10:28:48.127838 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.127827    2564 server.go:1268] "Started kubelet"
Jul 29 10:28:48.128018 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.127908    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 10:28:48.128018 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.127953    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 10:28:48.128154 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.128040    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 10:28:48.128663 ip-10-0-133-233 systemd[1]: Started Kubernetes Kubelet.
Jul 29 10:28:48.129926 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.129906    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 10:28:48.130515 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.130503    2564 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 10:28:48.135295 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.135270    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 10:28:48.137411 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.137386    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-884pg"
Jul 29 10:28:48.138854 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.138837    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-233.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 10:28:48.139880 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.138792    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-233.ec2.internal.18c6bbecd320d50b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-233.ec2.internal,UID:ip-10-0-133-233.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-233.ec2.internal,},FirstTimestamp:2026-07-29 10:28:48.127800587 +0000 UTC m=+0.378511285,LastTimestamp:2026-07-29 10:28:48.127800587 +0000 UTC m=+0.378511285,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-233.ec2.internal,}"
Jul 29 10:28:48.142586 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.142569    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 10:28:48.142586 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.142579    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 10:28:48.142713 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.142695    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-233.ec2.internal\" not found"
Jul 29 10:28:48.144316 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.144076    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 10:28:48.144316 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.144098    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 10:28:48.144316 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.144110    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 10:28:48.144316 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.144254    2564 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 10:28:48.144316 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.144261    2564 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 10:28:48.144664 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.144640    2564 factory.go:55] Registering systemd factory
Jul 29 10:28:48.144664 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.144659    2564 factory.go:223] Registration of the systemd container factory successfully
Jul 29 10:28:48.144886 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.144865    2564 factory.go:153] Registering CRI-O factory
Jul 29 10:28:48.144886 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.144884    2564 factory.go:223] Registration of the crio container factory successfully
Jul 29 10:28:48.145027 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.144950    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 10:28:48.145027 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.144980    2564 factory.go:103] Registering Raw factory
Jul 29 10:28:48.145027 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.144999    2564 manager.go:1196] Started watching for new ooms in manager
Jul 29 10:28:48.145166 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.145136    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 10:28:48.145564 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.145549    2564 manager.go:349] Starting recovery of all containers
Jul 29 10:28:48.148231 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.148200    2564 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-233.ec2.internal\" not found" node="ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.157177 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.157161    2564 manager.go:354] Recovery completed
Jul 29 10:28:48.161394 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.161380    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 10:28:48.163589 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.163574    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 10:28:48.163663 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.163600    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 10:28:48.163663 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.163610    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasSufficientPID"
Jul 29 10:28:48.164078 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.164066    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 10:28:48.164132 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.164077    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 10:28:48.164132 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.164070    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 10:28:48.164132 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.164095    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 10:28:48.164827 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.164812    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 10:28:48.164910 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.164842    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 10:28:48.164910 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.164857    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasSufficientPID"
Jul 29 10:28:48.165694 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.165682    2564 policy_none.go:49] "None policy: Start"
Jul 29 10:28:48.165737 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.165698    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 10:28:48.165737 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.165710    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 10:28:48.166563 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.166552    2564 policy_none.go:47] "Start"
Jul 29 10:28:48.195093 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.195078    2564 manager.go:337] "Starting Device Plugin manager"
Jul 29 10:28:48.208166 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.195110    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 10:28:48.208166 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.195121    2564 server.go:87] "Starting device plugin registration server"
Jul 29 10:28:48.208166 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.195399    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 10:28:48.208166 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.195410    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 10:28:48.208166 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.195537    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 10:28:48.208166 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.195632    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 10:28:48.208166 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.195645    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 10:28:48.208166 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.196185    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 10:28:48.208166 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.196262    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-233.ec2.internal\" not found"
Jul 29 10:28:48.268797 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.268764    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 10:28:48.269988 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.269965    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 10:28:48.270086 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.270008    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 10:28:48.270086 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.270037    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 10:28:48.270086 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.270080    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 10:28:48.272677 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.272654    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 10:28:48.295691 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.295672    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 10:28:48.296655 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.296639    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 10:28:48.296733 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.296671    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 10:28:48.296733 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.296682    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasSufficientPID"
Jul 29 10:28:48.296733 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.296714    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.304948 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.304908    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.304948 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.304930    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-233.ec2.internal\": node \"ip-10-0-133-233.ec2.internal\" not found"
Jul 29 10:28:48.331197 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.331172    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-233.ec2.internal\" not found"
Jul 29 10:28:48.370532 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.370506    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-233.ec2.internal"]
Jul 29 10:28:48.370628 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.370575    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 10:28:48.371493 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.371476    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 10:28:48.371547 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.371510    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 10:28:48.371547 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.371526    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasSufficientPID"
Jul 29 10:28:48.372835 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.372823    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 10:28:48.373003 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.372990    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.373037 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.373021    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 10:28:48.373640 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.373625    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 10:28:48.373698 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.373652    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 10:28:48.373698 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.373662    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasSufficientPID"
Jul 29 10:28:48.373698 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.373628    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 10:28:48.373779 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.373707    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 10:28:48.373779 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.373720    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasSufficientPID"
Jul 29 10:28:48.374987 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.374972    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.375078 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.374998    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 10:28:48.376438 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.376422    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 10:28:48.376519 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.376469    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 10:28:48.376519 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.376485    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeHasSufficientPID"
Jul 29 10:28:48.396973 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.396950    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-233.ec2.internal\" not found" node="ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.401431 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.401414    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-233.ec2.internal\" not found" node="ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.432052 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.432026    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-233.ec2.internal\" not found"
Jul 29 10:28:48.445564 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.445546    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/49addc5258e12ab015218ff440e6948d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal\" (UID: \"49addc5258e12ab015218ff440e6948d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.445658 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.445572    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49addc5258e12ab015218ff440e6948d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal\" (UID: \"49addc5258e12ab015218ff440e6948d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.445658 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.445592    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ee950c7e37cd6646704499e83e8cb37d-config\") pod \"kube-apiserver-proxy-ip-10-0-133-233.ec2.internal\" (UID: \"ee950c7e37cd6646704499e83e8cb37d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.532909 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.532860    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-233.ec2.internal\" not found"
Jul 29 10:28:48.546275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.546250    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ee950c7e37cd6646704499e83e8cb37d-config\") pod \"kube-apiserver-proxy-ip-10-0-133-233.ec2.internal\" (UID: \"ee950c7e37cd6646704499e83e8cb37d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.546353 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.546281    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/49addc5258e12ab015218ff440e6948d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal\" (UID: \"49addc5258e12ab015218ff440e6948d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.546353 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.546300    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49addc5258e12ab015218ff440e6948d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal\" (UID: \"49addc5258e12ab015218ff440e6948d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.546484 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.546369    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ee950c7e37cd6646704499e83e8cb37d-config\") pod \"kube-apiserver-proxy-ip-10-0-133-233.ec2.internal\" (UID: \"ee950c7e37cd6646704499e83e8cb37d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.546484 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.546422    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49addc5258e12ab015218ff440e6948d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal\" (UID: \"49addc5258e12ab015218ff440e6948d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.546554 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.546502    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/49addc5258e12ab015218ff440e6948d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal\" (UID: \"49addc5258e12ab015218ff440e6948d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.633732 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.633661    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-233.ec2.internal\" not found"
Jul 29 10:28:48.700198 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.700174    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.704939 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:48.704921    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-233.ec2.internal"
Jul 29 10:28:48.733898 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.733864    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-233.ec2.internal\" not found"
Jul 29 10:28:48.834521 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:48.834480    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-233.ec2.internal\" not found"
Jul 29 10:28:50.002418 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:50.002394    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-233.ec2.internal\" not found"
Jul 29 10:28:50.081734 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.081704    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 10:28:50.111055 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.111027    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal"
Jul 29 10:28:50.122246 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.122231    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 10:28:50.124478 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.124462    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-233.ec2.internal"
Jul 29 10:28:50.133782 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.133767    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 10:28:50.147929 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.147912    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 10:28:50.148066 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.148025    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 10:28:50.148109 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.148029    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 10:28:50.148109 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.148033    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 10:28:50.183213 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.183187    2564 apiserver.go:52] "Watching apiserver"
Jul 29 10:28:50.200509 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.200483    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 10:28:50.200861 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.200841    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-rvqcw","kube-system/konnectivity-agent-dclkn","kube-system/kube-apiserver-proxy-ip-10-0-133-233.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4","openshift-dns/node-resolver-wgrdq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal","openshift-multus/multus-additional-cni-plugins-rjgvn","openshift-multus/multus-mtx8j","openshift-cluster-node-tuning-operator/tuned-9jjlf","openshift-image-registry/node-ca-j5xql","openshift-multus/network-metrics-daemon-x5nh2","openshift-network-diagnostics/network-check-target-fp5xd","openshift-network-operator/iptables-alerter-2mz7m","openshift-ovn-kubernetes/ovnkube-node-psdkf"]
Jul 29 10:28:50.202362 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.202342    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.203681 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.203663    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2mz7m"
Jul 29 10:28:50.204963 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.204943    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 10:28:50.205053 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.204996    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 10:28:50.205111 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.205052    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 10:28:50.205233 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.205215    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.205233 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.205228    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 10:28:50.205331 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.205286    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 10:28:50.205331 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.205292    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jknfd\""
Jul 29 10:28:50.206308 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.206048    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 10:28:50.206401 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.206315    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lhg9x\""
Jul 29 10:28:50.206458 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.206403    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 10:28:50.206458 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.206407    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 10:28:50.206671 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.206654    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.207913 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.207874    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 10:28:50.208221 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.208200    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 10:28:50.208303 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.208219    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 10:28:50.208610 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.208592    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 10:28:50.208739 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.208708    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 10:28:50.209145 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.208879    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 10:28:50.209145 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.208990    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9mv6h\""
Jul 29 10:28:50.209285 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.209182    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 10:23:49 +0000 UTC" deadline="2028-04-02 17:58:05.114480558 +0000 UTC"
Jul 29 10:28:50.209285 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.209230    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14719h29m14.905254991s"
Jul 29 10:28:50.209285 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.209240    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wgrdq"
Jul 29 10:28:50.209431 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.209408    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-db55q\""
Jul 29 10:28:50.209431 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.209421    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 10:28:50.210203 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.210128    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 10:28:50.210390 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.210368    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 10:28:50.210469 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.210416    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 10:28:50.211302 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.211289    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.212550 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.212530    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 10:28:50.212654 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.212578    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lfc24\""
Jul 29 10:28:50.212654 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.212537    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.212829 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.212530    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 10:28:50.213360 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.213348    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-psxx7\""
Jul 29 10:28:50.213403 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.213363    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 10:28:50.213710 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.213697    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:50.213793 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:50.213747    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:28:50.214763 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.214749    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 10:28:50.214880 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.214829    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7996h\""
Jul 29 10:28:50.214999 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.214985    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 10:28:50.215073 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.215061    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dclkn"
Jul 29 10:28:50.216393 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216377    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j5xql"
Jul 29 10:28:50.216777 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216634    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.216777 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216668    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-run-ovn-kubernetes\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.216777 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216695    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-multus-cni-dir\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.216777 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216719    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-hostroot\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.216777 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216740    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-system-cni-dir\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.216777 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216755    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.216777 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216770    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-run-openvswitch\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.217228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216783    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-ovnkube-config\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.217228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216822    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.217228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216844    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ad2e5c5-ef58-4346-90c9-6abcab412125-tmp\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.217228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216862    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-var-lib-cni-bin\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.217228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216875    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-sysctl-d\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.217228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216901    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-systemd\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.217228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216934    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-slash\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.217228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216968    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-etc-selinux\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.217228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.216985    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-var-lib-openvswitch\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.217228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217001    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-run-ovn\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.217228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217025    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8zgl\" (UniqueName: \"kubernetes.io/projected/985d7ab1-89e7-4f96-908f-559b300397ea-kube-api-access-l8zgl\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.217228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217046    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-cnibin\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.217228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217070    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c9bh\" (UniqueName: \"kubernetes.io/projected/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-kube-api-access-7c9bh\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.217228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217092    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-cnibin\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.217228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-run-k8s-cni-cncf-io\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.217228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217137    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-kubernetes\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217156    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-tuned\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217178    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-ovnkube-script-lib\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217201    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz8ls\" (UniqueName: \"kubernetes.io/projected/5540a8cd-8b5e-496c-a681-d725efc6aa87-kube-api-access-dz8ls\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217221    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-etc-kubernetes\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217244    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9ckv\" (UniqueName: \"kubernetes.io/projected/0ad2e5c5-ef58-4346-90c9-6abcab412125-kube-api-access-b9ckv\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217268    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7czn\" (UniqueName: \"kubernetes.io/projected/7d9c53a1-f5af-4bfc-81ba-4703f2fc039b-kube-api-access-q7czn\") pod \"node-resolver-wgrdq\" (UID: \"7d9c53a1-f5af-4bfc-81ba-4703f2fc039b\") " pod="openshift-dns/node-resolver-wgrdq"
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217295    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-os-release\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217317    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-systemd-units\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217358    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217339    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-node-log\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217417    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-env-overrides\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217441    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-socket-dir\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217478    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-device-dir\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217501    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/985d7ab1-89e7-4f96-908f-559b300397ea-multus-daemon-config\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217523    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7d9c53a1-f5af-4bfc-81ba-4703f2fc039b-hosts-file\") pod \"node-resolver-wgrdq\" (UID: \"7d9c53a1-f5af-4bfc-81ba-4703f2fc039b\") " pod="openshift-dns/node-resolver-wgrdq"
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217547    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c8eca8a0-11a6-488c-8856-53c63d71c5bf-iptables-alerter-script\") pod \"iptables-alerter-2mz7m\" (UID: \"c8eca8a0-11a6-488c-8856-53c63d71c5bf\") " pod="openshift-network-operator/iptables-alerter-2mz7m"
Jul 29 10:28:50.218275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217558    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217572    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-kubelet\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217594    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-cni-bin\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217616    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2qlc\" (UniqueName: \"kubernetes.io/projected/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-kube-api-access-x2qlc\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217628    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dpbsw\""
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217638    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-sysconfig\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217661    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-sysctl-conf\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217705    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217738    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-registration-dir\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217762    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/985d7ab1-89e7-4f96-908f-559b300397ea-cni-binary-copy\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217819    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7jhr\" (UniqueName: \"kubernetes.io/projected/c8eca8a0-11a6-488c-8856-53c63d71c5bf-kube-api-access-z7jhr\") pod \"iptables-alerter-2mz7m\" (UID: \"c8eca8a0-11a6-488c-8856-53c63d71c5bf\") " pod="openshift-network-operator/iptables-alerter-2mz7m"
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217859    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-log-socket\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217864    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217897    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-cni-netd\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217922    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-ovn-node-metrics-cert\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:50.217923    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217945    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-multus-socket-dir-parent\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.219125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217970    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-host\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.217993    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-sys-fs\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218007    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-run-multus-certs\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218046    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-os-release\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218059    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-run-netns\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218072    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-lib-modules\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8eca8a0-11a6-488c-8856-53c63d71c5bf-host-slash\") pod \"iptables-alerter-2mz7m\" (UID: \"c8eca8a0-11a6-488c-8856-53c63d71c5bf\") " pod="openshift-network-operator/iptables-alerter-2mz7m"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218118    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-etc-openvswitch\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218133    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218156    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-var-lib-kubelet\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218171    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-modprobe-d\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218190    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-system-cni-dir\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218248    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-multus-conf-dir\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218263    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-run\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218285    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-sys\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218304    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-var-lib-kubelet\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218320    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d9c53a1-f5af-4bfc-81ba-4703f2fc039b-tmp-dir\") pod \"node-resolver-wgrdq\" (UID: \"7d9c53a1-f5af-4bfc-81ba-4703f2fc039b\") " pod="openshift-dns/node-resolver-wgrdq"
Jul 29 10:28:50.219697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218347    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-run-netns\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.220379 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218373    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-run-systemd\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.220379 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218391    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-var-lib-cni-multus\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.220379 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218417    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-cni-binary-copy\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.220379 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218649    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vmcxw\""
Jul 29 10:28:50.220379 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218724    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 10:28:50.220379 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218726    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 10:28:50.220379 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.218879    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 10:28:50.220379 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.219002    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:28:50.220379 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:50.219047    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:28:50.222120 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.222102    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 10:28:50.238337 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.238321    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mrghn"
Jul 29 10:28:50.246764 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.246719    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mrghn"
Jul 29 10:28:50.313320 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.313296    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 10:28:50.318695 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318549    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a6d54b5d-72a9-412c-8dbc-e4993832122a-dbus\") pod \"global-pull-secret-syncer-rvqcw\" (UID: \"a6d54b5d-72a9-412c-8dbc-e4993832122a\") " pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:50.318695 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318583    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-system-cni-dir\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.318695 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318615    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.318695 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318640    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-run-openvswitch\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.318695 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318664    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-ovnkube-config\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.318695 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318672    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-system-cni-dir\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.319026 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318685    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.319026 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318746    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.319026 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318749    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ad2e5c5-ef58-4346-90c9-6abcab412125-tmp\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.319026 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318782    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs\") pod \"network-metrics-daemon-x5nh2\" (UID: \"d598bd41-0d98-42fb-b082-8582a3b3b5ef\") " pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:28:50.319026 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318824    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p4vp\" (UniqueName: \"kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp\") pod \"network-check-target-fp5xd\" (UID: \"33f96a45-1e06-4af4-bcbe-7e494cfcbfe2\") " pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:28:50.319026 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318854    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-var-lib-cni-bin\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.319026 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318878    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-sysctl-d\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.319026 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318901    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-systemd\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.319026 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318927    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a6d54b5d-72a9-412c-8dbc-e4993832122a-kubelet-config\") pod \"global-pull-secret-syncer-rvqcw\" (UID: \"a6d54b5d-72a9-412c-8dbc-e4993832122a\") " pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:50.319026 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318944    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-var-lib-cni-bin\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.319026 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318954    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-slash\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.319026 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.318995    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-etc-selinux\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.319026 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319017    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-systemd\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319064    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-slash\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319104    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319136    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-sysctl-d\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319019    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-var-lib-openvswitch\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319168    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-etc-selinux\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319178    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-var-lib-openvswitch\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319190    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-run-ovn\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319213    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-run-openvswitch\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8zgl\" (UniqueName: \"kubernetes.io/projected/985d7ab1-89e7-4f96-908f-559b300397ea-kube-api-access-l8zgl\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319240    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-cnibin\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319242    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-run-ovn\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319265    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c9bh\" (UniqueName: \"kubernetes.io/projected/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-kube-api-access-7c9bh\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319286    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-cnibin\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319289    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-cnibin\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319313    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-run-k8s-cni-cncf-io\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319333    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-kubernetes\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319354    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-tuned\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.319643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319360    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-ovnkube-config\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319380    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9a04cac1-c3f9-42db-9339-54caac308ad2-konnectivity-ca\") pod \"konnectivity-agent-dclkn\" (UID: \"9a04cac1-c3f9-42db-9339-54caac308ad2\") " pod="kube-system/konnectivity-agent-dclkn"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319404    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-ovnkube-script-lib\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-run-k8s-cni-cncf-io\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319432    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz8ls\" (UniqueName: \"kubernetes.io/projected/5540a8cd-8b5e-496c-a681-d725efc6aa87-kube-api-access-dz8ls\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319441    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-kubernetes\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319380    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-cnibin\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319455    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-etc-kubernetes\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319495    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9ckv\" (UniqueName: \"kubernetes.io/projected/0ad2e5c5-ef58-4346-90c9-6abcab412125-kube-api-access-b9ckv\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319501    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-etc-kubernetes\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319538    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7czn\" (UniqueName: \"kubernetes.io/projected/7d9c53a1-f5af-4bfc-81ba-4703f2fc039b-kube-api-access-q7czn\") pod \"node-resolver-wgrdq\" (UID: \"7d9c53a1-f5af-4bfc-81ba-4703f2fc039b\") " pod="openshift-dns/node-resolver-wgrdq"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319557    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319564    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret\") pod \"global-pull-secret-syncer-rvqcw\" (UID: \"a6d54b5d-72a9-412c-8dbc-e4993832122a\") " pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319589    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-os-release\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319614    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-systemd-units\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319649    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-node-log\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319666    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-systemd-units\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.320513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319675    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-env-overrides\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319726    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-os-release\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319764    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-node-log\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319815    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-socket-dir\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319852    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-device-dir\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319876    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/985d7ab1-89e7-4f96-908f-559b300397ea-multus-daemon-config\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319900    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7d9c53a1-f5af-4bfc-81ba-4703f2fc039b-hosts-file\") pod \"node-resolver-wgrdq\" (UID: \"7d9c53a1-f5af-4bfc-81ba-4703f2fc039b\") " pod="openshift-dns/node-resolver-wgrdq"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319923    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c8eca8a0-11a6-488c-8856-53c63d71c5bf-iptables-alerter-script\") pod \"iptables-alerter-2mz7m\" (UID: \"c8eca8a0-11a6-488c-8856-53c63d71c5bf\") " pod="openshift-network-operator/iptables-alerter-2mz7m"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319944    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-kubelet\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319954    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-socket-dir\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319963    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-cni-bin\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.319983    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2qlc\" (UniqueName: \"kubernetes.io/projected/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-kube-api-access-x2qlc\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320003    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7d9c53a1-f5af-4bfc-81ba-4703f2fc039b-hosts-file\") pod \"node-resolver-wgrdq\" (UID: \"7d9c53a1-f5af-4bfc-81ba-4703f2fc039b\") " pod="openshift-dns/node-resolver-wgrdq"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320006    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-sysconfig\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320025    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-sysctl-conf\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320030    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-device-dir\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320049    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9a04cac1-c3f9-42db-9339-54caac308ad2-agent-certs\") pod \"konnectivity-agent-dclkn\" (UID: \"9a04cac1-c3f9-42db-9339-54caac308ad2\") " pod="kube-system/konnectivity-agent-dclkn"
Jul 29 10:28:50.321340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320069    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3ffc02ea-68ae-4d60-82a4-01d159cd668c-serviceca\") pod \"node-ca-j5xql\" (UID: \"3ffc02ea-68ae-4d60-82a4-01d159cd668c\") " pod="openshift-image-registry/node-ca-j5xql"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320067    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-ovnkube-script-lib\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320087    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-env-overrides\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320089    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320140    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-sysconfig\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320181    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-registration-dir\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320201    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-sysctl-conf\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320204    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/985d7ab1-89e7-4f96-908f-559b300397ea-cni-binary-copy\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320243    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7jhr\" (UniqueName: \"kubernetes.io/projected/c8eca8a0-11a6-488c-8856-53c63d71c5bf-kube-api-access-z7jhr\") pod \"iptables-alerter-2mz7m\" (UID: \"c8eca8a0-11a6-488c-8856-53c63d71c5bf\") " pod="openshift-network-operator/iptables-alerter-2mz7m"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320271    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-log-socket\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320316    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-cni-netd\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320341    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-ovn-node-metrics-cert\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320366    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-multus-socket-dir-parent\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320392    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-host\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320420    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz6w5\" (UniqueName: \"kubernetes.io/projected/3ffc02ea-68ae-4d60-82a4-01d159cd668c-kube-api-access-kz6w5\") pod \"node-ca-j5xql\" (UID: \"3ffc02ea-68ae-4d60-82a4-01d159cd668c\") " pod="openshift-image-registry/node-ca-j5xql"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320452    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-sys-fs\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320477    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-kubelet\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.322190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320478    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-run-multus-certs\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320518    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-run-multus-certs\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320519    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2f7t\" (UniqueName: \"kubernetes.io/projected/d598bd41-0d98-42fb-b082-8582a3b3b5ef-kube-api-access-p2f7t\") pod \"network-metrics-daemon-x5nh2\" (UID: \"d598bd41-0d98-42fb-b082-8582a3b3b5ef\") " pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320560    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-log-socket\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320585    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/985d7ab1-89e7-4f96-908f-559b300397ea-cni-binary-copy\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320643    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-registration-dir\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320699    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-cni-bin\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320752    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ffc02ea-68ae-4d60-82a4-01d159cd668c-host\") pod \"node-ca-j5xql\" (UID: \"3ffc02ea-68ae-4d60-82a4-01d159cd668c\") " pod="openshift-image-registry/node-ca-j5xql"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320769    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-multus-socket-dir-parent\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320788    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-os-release\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320851    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-host\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320893    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-run-netns\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320924    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-lib-modules\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320942    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5540a8cd-8b5e-496c-a681-d725efc6aa87-sys-fs\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.320967    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-cni-netd\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321004    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8eca8a0-11a6-488c-8856-53c63d71c5bf-host-slash\") pod \"iptables-alerter-2mz7m\" (UID: \"c8eca8a0-11a6-488c-8856-53c63d71c5bf\") " pod="openshift-network-operator/iptables-alerter-2mz7m"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321012    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321029    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-etc-openvswitch\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.322956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321022    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-os-release\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321044    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-lib-modules\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321062    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-run-netns\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321097    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8eca8a0-11a6-488c-8856-53c63d71c5bf-host-slash\") pod \"iptables-alerter-2mz7m\" (UID: \"c8eca8a0-11a6-488c-8856-53c63d71c5bf\") " pod="openshift-network-operator/iptables-alerter-2mz7m"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321107    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321134    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-etc-openvswitch\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321171    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-var-lib-kubelet\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321238    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-modprobe-d\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321255    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-var-lib-kubelet\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321293    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c8eca8a0-11a6-488c-8856-53c63d71c5bf-iptables-alerter-script\") pod \"iptables-alerter-2mz7m\" (UID: \"c8eca8a0-11a6-488c-8856-53c63d71c5bf\") " pod="openshift-network-operator/iptables-alerter-2mz7m"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321277    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-system-cni-dir\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321294    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/985d7ab1-89e7-4f96-908f-559b300397ea-multus-daemon-config\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321327    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-multus-conf-dir\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321334    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-system-cni-dir\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321361    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-multus-conf-dir\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321397    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-run\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321423    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-modprobe-d\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.323456 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321432    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-sys\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321456    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-run\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321478    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-var-lib-kubelet\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321494    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-sys\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321504    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d9c53a1-f5af-4bfc-81ba-4703f2fc039b-tmp-dir\") pod \"node-resolver-wgrdq\" (UID: \"7d9c53a1-f5af-4bfc-81ba-4703f2fc039b\") " pod="openshift-dns/node-resolver-wgrdq"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321529    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-run-netns\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321540    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ad2e5c5-ef58-4346-90c9-6abcab412125-var-lib-kubelet\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321552    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-run-systemd\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321574    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-var-lib-cni-multus\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321584    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-run-netns\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321597    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-cni-binary-copy\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321635    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321637    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-run-systemd\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321661    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-run-ovn-kubernetes\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321679    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-host-var-lib-cni-multus\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321684    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-multus-cni-dir\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321706    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-hostroot\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321772    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-hostroot\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.323974 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321826    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-host-run-ovn-kubernetes\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.324464 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321921    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/985d7ab1-89e7-4f96-908f-559b300397ea-multus-cni-dir\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.324464 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.321945    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d9c53a1-f5af-4bfc-81ba-4703f2fc039b-tmp-dir\") pod \"node-resolver-wgrdq\" (UID: \"7d9c53a1-f5af-4bfc-81ba-4703f2fc039b\") " pod="openshift-dns/node-resolver-wgrdq"
Jul 29 10:28:50.324464 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.322149    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.324464 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.322229    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-cni-binary-copy\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.324464 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.322903    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ad2e5c5-ef58-4346-90c9-6abcab412125-tmp\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.324464 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.322976    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0ad2e5c5-ef58-4346-90c9-6abcab412125-etc-tuned\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.324464 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.323117    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-ovn-node-metrics-cert\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.328165 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.328119    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c9bh\" (UniqueName: \"kubernetes.io/projected/dbf16cf7-3ffb-4dd6-a559-0d47817db5cf-kube-api-access-7c9bh\") pod \"multus-additional-cni-plugins-rjgvn\" (UID: \"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf\") " pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.328369 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.328329    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7czn\" (UniqueName: \"kubernetes.io/projected/7d9c53a1-f5af-4bfc-81ba-4703f2fc039b-kube-api-access-q7czn\") pod \"node-resolver-wgrdq\" (UID: \"7d9c53a1-f5af-4bfc-81ba-4703f2fc039b\") " pod="openshift-dns/node-resolver-wgrdq"
Jul 29 10:28:50.328495 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.328474    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8zgl\" (UniqueName: \"kubernetes.io/projected/985d7ab1-89e7-4f96-908f-559b300397ea-kube-api-access-l8zgl\") pod \"multus-mtx8j\" (UID: \"985d7ab1-89e7-4f96-908f-559b300397ea\") " pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.328568 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.328504    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9ckv\" (UniqueName: \"kubernetes.io/projected/0ad2e5c5-ef58-4346-90c9-6abcab412125-kube-api-access-b9ckv\") pod \"tuned-9jjlf\" (UID: \"0ad2e5c5-ef58-4346-90c9-6abcab412125\") " pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.328568 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.328519    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz8ls\" (UniqueName: \"kubernetes.io/projected/5540a8cd-8b5e-496c-a681-d725efc6aa87-kube-api-access-dz8ls\") pod \"aws-ebs-csi-driver-node-9t9z4\" (UID: \"5540a8cd-8b5e-496c-a681-d725efc6aa87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.328705 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.328689    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7jhr\" (UniqueName: \"kubernetes.io/projected/c8eca8a0-11a6-488c-8856-53c63d71c5bf-kube-api-access-z7jhr\") pod \"iptables-alerter-2mz7m\" (UID: \"c8eca8a0-11a6-488c-8856-53c63d71c5bf\") " pod="openshift-network-operator/iptables-alerter-2mz7m"
Jul 29 10:28:50.328768 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.328727    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2qlc\" (UniqueName: \"kubernetes.io/projected/1ab4b50a-79d2-43f1-b4c0-e7459adb20c7-kube-api-access-x2qlc\") pod \"ovnkube-node-psdkf\" (UID: \"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.369159 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.368886    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 10:28:50.369958 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:50.369912    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49addc5258e12ab015218ff440e6948d.slice/crio-047fe0c47a5604ce93a8d1c2b0176407357f05983f470095c73ad56ec47a07af WatchSource:0}: Error finding container 047fe0c47a5604ce93a8d1c2b0176407357f05983f470095c73ad56ec47a07af: Status 404 returned error can't find the container with id 047fe0c47a5604ce93a8d1c2b0176407357f05983f470095c73ad56ec47a07af
Jul 29 10:28:50.370387 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:50.370367    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee950c7e37cd6646704499e83e8cb37d.slice/crio-40f70100b29fdfa6ca811cd4fd86734d93907aca9c0dbfb19b7d00dbd406a9bc WatchSource:0}: Error finding container 40f70100b29fdfa6ca811cd4fd86734d93907aca9c0dbfb19b7d00dbd406a9bc: Status 404 returned error can't find the container with id 40f70100b29fdfa6ca811cd4fd86734d93907aca9c0dbfb19b7d00dbd406a9bc
Jul 29 10:28:50.374662 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.374642    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 10:28:50.422489 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.422466    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret\") pod \"global-pull-secret-syncer-rvqcw\" (UID: \"a6d54b5d-72a9-412c-8dbc-e4993832122a\") " pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:50.422604 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.422498    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9a04cac1-c3f9-42db-9339-54caac308ad2-agent-certs\") pod \"konnectivity-agent-dclkn\" (UID: \"9a04cac1-c3f9-42db-9339-54caac308ad2\") " pod="kube-system/konnectivity-agent-dclkn"
Jul 29 10:28:50.422604 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.422516    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3ffc02ea-68ae-4d60-82a4-01d159cd668c-serviceca\") pod \"node-ca-j5xql\" (UID: \"3ffc02ea-68ae-4d60-82a4-01d159cd668c\") " pod="openshift-image-registry/node-ca-j5xql"
Jul 29 10:28:50.422604 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.422541    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz6w5\" (UniqueName: \"kubernetes.io/projected/3ffc02ea-68ae-4d60-82a4-01d159cd668c-kube-api-access-kz6w5\") pod \"node-ca-j5xql\" (UID: \"3ffc02ea-68ae-4d60-82a4-01d159cd668c\") " pod="openshift-image-registry/node-ca-j5xql"
Jul 29 10:28:50.422604 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.422565    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2f7t\" (UniqueName: \"kubernetes.io/projected/d598bd41-0d98-42fb-b082-8582a3b3b5ef-kube-api-access-p2f7t\") pod \"network-metrics-daemon-x5nh2\" (UID: \"d598bd41-0d98-42fb-b082-8582a3b3b5ef\") " pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:28:50.422604 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.422585    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ffc02ea-68ae-4d60-82a4-01d159cd668c-host\") pod \"node-ca-j5xql\" (UID: \"3ffc02ea-68ae-4d60-82a4-01d159cd668c\") " pod="openshift-image-registry/node-ca-j5xql"
Jul 29 10:28:50.422886 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.422620    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a6d54b5d-72a9-412c-8dbc-e4993832122a-dbus\") pod \"global-pull-secret-syncer-rvqcw\" (UID: \"a6d54b5d-72a9-412c-8dbc-e4993832122a\") " pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:50.422886 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:50.422623    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:50.422886 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.422664    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ffc02ea-68ae-4d60-82a4-01d159cd668c-host\") pod \"node-ca-j5xql\" (UID: \"3ffc02ea-68ae-4d60-82a4-01d159cd668c\") " pod="openshift-image-registry/node-ca-j5xql"
Jul 29 10:28:50.422886 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:50.422693    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret podName:a6d54b5d-72a9-412c-8dbc-e4993832122a nodeName:}" failed. No retries permitted until 2026-07-29 10:28:50.922671429 +0000 UTC m=+2.106027835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret") pod "global-pull-secret-syncer-rvqcw" (UID: "a6d54b5d-72a9-412c-8dbc-e4993832122a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:50.422886 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.422710    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs\") pod \"network-metrics-daemon-x5nh2\" (UID: \"d598bd41-0d98-42fb-b082-8582a3b3b5ef\") " pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:28:50.422886 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.422727    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p4vp\" (UniqueName: \"kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp\") pod \"network-check-target-fp5xd\" (UID: \"33f96a45-1e06-4af4-bcbe-7e494cfcbfe2\") " pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:28:50.422886 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.422746    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a6d54b5d-72a9-412c-8dbc-e4993832122a-kubelet-config\") pod \"global-pull-secret-syncer-rvqcw\" (UID: \"a6d54b5d-72a9-412c-8dbc-e4993832122a\") " pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:50.422886 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.422776    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9a04cac1-c3f9-42db-9339-54caac308ad2-konnectivity-ca\") pod \"konnectivity-agent-dclkn\" (UID: \"9a04cac1-c3f9-42db-9339-54caac308ad2\") " pod="kube-system/konnectivity-agent-dclkn"
Jul 29 10:28:50.422886 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.422742    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a6d54b5d-72a9-412c-8dbc-e4993832122a-dbus\") pod \"global-pull-secret-syncer-rvqcw\" (UID: \"a6d54b5d-72a9-412c-8dbc-e4993832122a\") " pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:50.423280 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.422898    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a6d54b5d-72a9-412c-8dbc-e4993832122a-kubelet-config\") pod \"global-pull-secret-syncer-rvqcw\" (UID: \"a6d54b5d-72a9-412c-8dbc-e4993832122a\") " pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:50.423280 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:50.422942    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:50.423280 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:50.422995    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs podName:d598bd41-0d98-42fb-b082-8582a3b3b5ef nodeName:}" failed. No retries permitted until 2026-07-29 10:28:50.922978479 +0000 UTC m=+2.106334878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs") pod "network-metrics-daemon-x5nh2" (UID: "d598bd41-0d98-42fb-b082-8582a3b3b5ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:50.423280 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.423133    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3ffc02ea-68ae-4d60-82a4-01d159cd668c-serviceca\") pod \"node-ca-j5xql\" (UID: \"3ffc02ea-68ae-4d60-82a4-01d159cd668c\") " pod="openshift-image-registry/node-ca-j5xql"
Jul 29 10:28:50.423280 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.423272    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9a04cac1-c3f9-42db-9339-54caac308ad2-konnectivity-ca\") pod \"konnectivity-agent-dclkn\" (UID: \"9a04cac1-c3f9-42db-9339-54caac308ad2\") " pod="kube-system/konnectivity-agent-dclkn"
Jul 29 10:28:50.424939 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.424920    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9a04cac1-c3f9-42db-9339-54caac308ad2-agent-certs\") pod \"konnectivity-agent-dclkn\" (UID: \"9a04cac1-c3f9-42db-9339-54caac308ad2\") " pod="kube-system/konnectivity-agent-dclkn"
Jul 29 10:28:50.431644 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:50.431615    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 10:28:50.431644 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:50.431632    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 10:28:50.431644 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:50.431641    2564 projected.go:196] Error preparing data for projected volume kube-api-access-7p4vp for pod openshift-network-diagnostics/network-check-target-fp5xd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:50.431878 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:50.431681    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp podName:33f96a45-1e06-4af4-bcbe-7e494cfcbfe2 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:50.931670979 +0000 UTC m=+2.115027388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7p4vp" (UniqueName: "kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp") pod "network-check-target-fp5xd" (UID: "33f96a45-1e06-4af4-bcbe-7e494cfcbfe2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:50.432892 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.432873    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2f7t\" (UniqueName: \"kubernetes.io/projected/d598bd41-0d98-42fb-b082-8582a3b3b5ef-kube-api-access-p2f7t\") pod \"network-metrics-daemon-x5nh2\" (UID: \"d598bd41-0d98-42fb-b082-8582a3b3b5ef\") " pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:28:50.433548 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.433529    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz6w5\" (UniqueName: \"kubernetes.io/projected/3ffc02ea-68ae-4d60-82a4-01d159cd668c-kube-api-access-kz6w5\") pod \"node-ca-j5xql\" (UID: \"3ffc02ea-68ae-4d60-82a4-01d159cd668c\") " pod="openshift-image-registry/node-ca-j5xql"
Jul 29 10:28:50.531654 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.531576    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rjgvn"
Jul 29 10:28:50.539604 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:50.539581    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbf16cf7_3ffb_4dd6_a559_0d47817db5cf.slice/crio-0063cbe9ed0eeea2648eb8c8a4b7d1fb86b415bf09c39ba7208a7fd1de3de707 WatchSource:0}: Error finding container 0063cbe9ed0eeea2648eb8c8a4b7d1fb86b415bf09c39ba7208a7fd1de3de707: Status 404 returned error can't find the container with id 0063cbe9ed0eeea2648eb8c8a4b7d1fb86b415bf09c39ba7208a7fd1de3de707
Jul 29 10:28:50.552602 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.552585    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2mz7m"
Jul 29 10:28:50.559438 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:50.559412    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8eca8a0_11a6_488c_8856_53c63d71c5bf.slice/crio-77c3056f350642b5a53463d8e24f3f1be81c19d694878fcaa9fd0aaf8e2a647f WatchSource:0}: Error finding container 77c3056f350642b5a53463d8e24f3f1be81c19d694878fcaa9fd0aaf8e2a647f: Status 404 returned error can't find the container with id 77c3056f350642b5a53463d8e24f3f1be81c19d694878fcaa9fd0aaf8e2a647f
Jul 29 10:28:50.573195 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.573176    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:28:50.577903 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.577885    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4"
Jul 29 10:28:50.580243 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:50.580225    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ab4b50a_79d2_43f1_b4c0_e7459adb20c7.slice/crio-c0f9202352636e59a8bc3ff5150f6b45819e7e17497be9fdcdc6adcf930576ed WatchSource:0}: Error finding container c0f9202352636e59a8bc3ff5150f6b45819e7e17497be9fdcdc6adcf930576ed: Status 404 returned error can't find the container with id c0f9202352636e59a8bc3ff5150f6b45819e7e17497be9fdcdc6adcf930576ed
Jul 29 10:28:50.584737 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:50.584719    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5540a8cd_8b5e_496c_a681_d725efc6aa87.slice/crio-e94df35308df7087837a7f80d0a4b5d6ea935175606cdb86a1b9c01f48f15a92 WatchSource:0}: Error finding container e94df35308df7087837a7f80d0a4b5d6ea935175606cdb86a1b9c01f48f15a92: Status 404 returned error can't find the container with id e94df35308df7087837a7f80d0a4b5d6ea935175606cdb86a1b9c01f48f15a92
Jul 29 10:28:50.597355 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.597335    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wgrdq"
Jul 29 10:28:50.604795 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:50.604774    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d9c53a1_f5af_4bfc_81ba_4703f2fc039b.slice/crio-96b328e69bbd2322209336a42c40f577a9a39cffd21077773a3175fc39af99ff WatchSource:0}: Error finding container 96b328e69bbd2322209336a42c40f577a9a39cffd21077773a3175fc39af99ff: Status 404 returned error can't find the container with id 96b328e69bbd2322209336a42c40f577a9a39cffd21077773a3175fc39af99ff
Jul 29 10:28:50.610622 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.610602    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mtx8j"
Jul 29 10:28:50.616348 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.616333    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9jjlf"
Jul 29 10:28:50.616598 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:50.616580    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod985d7ab1_89e7_4f96_908f_559b300397ea.slice/crio-d5e847d227c2a023cbd36de442c9c98837907f29692ab769434e71be381283e6 WatchSource:0}: Error finding container d5e847d227c2a023cbd36de442c9c98837907f29692ab769434e71be381283e6: Status 404 returned error can't find the container with id d5e847d227c2a023cbd36de442c9c98837907f29692ab769434e71be381283e6
Jul 29 10:28:50.623896 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:50.623876    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad2e5c5_ef58_4346_90c9_6abcab412125.slice/crio-228bebfaa568786b78093278d2c343155ba5a74ce8ad77fdf6116fa2add46232 WatchSource:0}: Error finding container 228bebfaa568786b78093278d2c343155ba5a74ce8ad77fdf6116fa2add46232: Status 404 returned error can't find the container with id 228bebfaa568786b78093278d2c343155ba5a74ce8ad77fdf6116fa2add46232
Jul 29 10:28:50.639946 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.639928    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dclkn"
Jul 29 10:28:50.644928 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.644913    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j5xql"
Jul 29 10:28:50.646835 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:50.646815    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a04cac1_c3f9_42db_9339_54caac308ad2.slice/crio-73bae11dce2270a9a16ac5d80ec103ec14f14a27781ba470e5407ae0ebdeaab4 WatchSource:0}: Error finding container 73bae11dce2270a9a16ac5d80ec103ec14f14a27781ba470e5407ae0ebdeaab4: Status 404 returned error can't find the container with id 73bae11dce2270a9a16ac5d80ec103ec14f14a27781ba470e5407ae0ebdeaab4
Jul 29 10:28:50.651600 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:28:50.651581    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ffc02ea_68ae_4d60_82a4_01d159cd668c.slice/crio-a8393b902128f2ad2268919e0dac1d58b81310b8bfa13314b7a83bb9a7b13f13 WatchSource:0}: Error finding container a8393b902128f2ad2268919e0dac1d58b81310b8bfa13314b7a83bb9a7b13f13: Status 404 returned error can't find the container with id a8393b902128f2ad2268919e0dac1d58b81310b8bfa13314b7a83bb9a7b13f13
Jul 29 10:28:50.926252 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.926178    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs\") pod \"network-metrics-daemon-x5nh2\" (UID: \"d598bd41-0d98-42fb-b082-8582a3b3b5ef\") " pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:28:50.926252 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:50.926246    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret\") pod \"global-pull-secret-syncer-rvqcw\" (UID: \"a6d54b5d-72a9-412c-8dbc-e4993832122a\") " pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:50.926447 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:50.926338    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:50.926447 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:50.926408    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs podName:d598bd41-0d98-42fb-b082-8582a3b3b5ef nodeName:}" failed. No retries permitted until 2026-07-29 10:28:51.926388007 +0000 UTC m=+3.109744415 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs") pod "network-metrics-daemon-x5nh2" (UID: "d598bd41-0d98-42fb-b082-8582a3b3b5ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:50.926447 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:50.926421    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:50.926560 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:50.926464    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret podName:a6d54b5d-72a9-412c-8dbc-e4993832122a nodeName:}" failed. No retries permitted until 2026-07-29 10:28:51.926452369 +0000 UTC m=+3.109808762 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret") pod "global-pull-secret-syncer-rvqcw" (UID: "a6d54b5d-72a9-412c-8dbc-e4993832122a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:51.027370 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.027332    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p4vp\" (UniqueName: \"kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp\") pod \"network-check-target-fp5xd\" (UID: \"33f96a45-1e06-4af4-bcbe-7e494cfcbfe2\") " pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:28:51.027538 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:51.027513    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 10:28:51.027538 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:51.027531    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 10:28:51.027647 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:51.027543    2564 projected.go:196] Error preparing data for projected volume kube-api-access-7p4vp for pod openshift-network-diagnostics/network-check-target-fp5xd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:51.027647 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:51.027599    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp podName:33f96a45-1e06-4af4-bcbe-7e494cfcbfe2 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:52.027580027 +0000 UTC m=+3.210936443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7p4vp" (UniqueName: "kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp") pod "network-check-target-fp5xd" (UID: "33f96a45-1e06-4af4-bcbe-7e494cfcbfe2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:51.249332 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.249223    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 10:23:50 +0000 UTC" deadline="2028-01-16 10:17:48.314246867 +0000 UTC"
Jul 29 10:28:51.249332 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.249288    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12863h48m57.064971321s"
Jul 29 10:28:51.340415 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.340385    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:51.340575 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:51.340517    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:28:51.348089 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.348033    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j5xql" event={"ID":"3ffc02ea-68ae-4d60-82a4-01d159cd668c","Type":"ContainerStarted","Data":"a8393b902128f2ad2268919e0dac1d58b81310b8bfa13314b7a83bb9a7b13f13"}
Jul 29 10:28:51.356487 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.356454    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dclkn" event={"ID":"9a04cac1-c3f9-42db-9339-54caac308ad2","Type":"ContainerStarted","Data":"73bae11dce2270a9a16ac5d80ec103ec14f14a27781ba470e5407ae0ebdeaab4"}
Jul 29 10:28:51.361138 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.361115    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 10:28:51.365023 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.364094    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mtx8j" event={"ID":"985d7ab1-89e7-4f96-908f-559b300397ea","Type":"ContainerStarted","Data":"d5e847d227c2a023cbd36de442c9c98837907f29692ab769434e71be381283e6"}
Jul 29 10:28:51.374191 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.374109    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wgrdq" event={"ID":"7d9c53a1-f5af-4bfc-81ba-4703f2fc039b","Type":"ContainerStarted","Data":"96b328e69bbd2322209336a42c40f577a9a39cffd21077773a3175fc39af99ff"}
Jul 29 10:28:51.379675 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.379643    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4" event={"ID":"5540a8cd-8b5e-496c-a681-d725efc6aa87","Type":"ContainerStarted","Data":"e94df35308df7087837a7f80d0a4b5d6ea935175606cdb86a1b9c01f48f15a92"}
Jul 29 10:28:51.382468 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.382443    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2mz7m" event={"ID":"c8eca8a0-11a6-488c-8856-53c63d71c5bf","Type":"ContainerStarted","Data":"77c3056f350642b5a53463d8e24f3f1be81c19d694878fcaa9fd0aaf8e2a647f"}
Jul 29 10:28:51.385417 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.385393    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-233.ec2.internal" event={"ID":"ee950c7e37cd6646704499e83e8cb37d","Type":"ContainerStarted","Data":"40f70100b29fdfa6ca811cd4fd86734d93907aca9c0dbfb19b7d00dbd406a9bc"}
Jul 29 10:28:51.388664 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.388641    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9jjlf" event={"ID":"0ad2e5c5-ef58-4346-90c9-6abcab412125","Type":"ContainerStarted","Data":"228bebfaa568786b78093278d2c343155ba5a74ce8ad77fdf6116fa2add46232"}
Jul 29 10:28:51.395446 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.395423    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf" event={"ID":"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7","Type":"ContainerStarted","Data":"c0f9202352636e59a8bc3ff5150f6b45819e7e17497be9fdcdc6adcf930576ed"}
Jul 29 10:28:51.401963 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.401938    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjgvn" event={"ID":"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf","Type":"ContainerStarted","Data":"0063cbe9ed0eeea2648eb8c8a4b7d1fb86b415bf09c39ba7208a7fd1de3de707"}
Jul 29 10:28:51.413278 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.413252    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal" event={"ID":"49addc5258e12ab015218ff440e6948d","Type":"ContainerStarted","Data":"047fe0c47a5604ce93a8d1c2b0176407357f05983f470095c73ad56ec47a07af"}
Jul 29 10:28:51.621423 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.621336    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 10:28:51.934890 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.934793    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret\") pod \"global-pull-secret-syncer-rvqcw\" (UID: \"a6d54b5d-72a9-412c-8dbc-e4993832122a\") " pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:51.935048 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:51.934894    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs\") pod \"network-metrics-daemon-x5nh2\" (UID: \"d598bd41-0d98-42fb-b082-8582a3b3b5ef\") " pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:28:51.935048 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:51.935028    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:51.935166 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:51.935086    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs podName:d598bd41-0d98-42fb-b082-8582a3b3b5ef nodeName:}" failed. No retries permitted until 2026-07-29 10:28:53.935067945 +0000 UTC m=+5.118424344 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs") pod "network-metrics-daemon-x5nh2" (UID: "d598bd41-0d98-42fb-b082-8582a3b3b5ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:51.935540 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:51.935505    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:51.935628 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:51.935558    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret podName:a6d54b5d-72a9-412c-8dbc-e4993832122a nodeName:}" failed. No retries permitted until 2026-07-29 10:28:53.935542959 +0000 UTC m=+5.118899358 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret") pod "global-pull-secret-syncer-rvqcw" (UID: "a6d54b5d-72a9-412c-8dbc-e4993832122a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:52.035241 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:52.035206    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p4vp\" (UniqueName: \"kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp\") pod \"network-check-target-fp5xd\" (UID: \"33f96a45-1e06-4af4-bcbe-7e494cfcbfe2\") " pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:28:52.035419 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:52.035390    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 10:28:52.035419 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:52.035412    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 10:28:52.035536 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:52.035425    2564 projected.go:196] Error preparing data for projected volume kube-api-access-7p4vp for pod openshift-network-diagnostics/network-check-target-fp5xd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:52.035536 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:52.035482    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp podName:33f96a45-1e06-4af4-bcbe-7e494cfcbfe2 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:54.035463735 +0000 UTC m=+5.218820143 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7p4vp" (UniqueName: "kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp") pod "network-check-target-fp5xd" (UID: "33f96a45-1e06-4af4-bcbe-7e494cfcbfe2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:52.250586 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:52.250475    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 10:23:50 +0000 UTC" deadline="2028-03-21 17:34:58.307021185 +0000 UTC"
Jul 29 10:28:52.250586 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:52.250536    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14431h6m6.056489187s"
Jul 29 10:28:52.340548 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:52.337717    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:28:52.340548 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:52.337853    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:28:52.340548 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:52.337904    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:28:52.340548 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:52.338020    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:28:53.102178 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:53.102142    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 10:28:53.345795 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:53.345769    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:53.346239 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:53.345905    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:28:53.951332 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:53.951261    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs\") pod \"network-metrics-daemon-x5nh2\" (UID: \"d598bd41-0d98-42fb-b082-8582a3b3b5ef\") " pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:28:53.951454 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:53.951336    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret\") pod \"global-pull-secret-syncer-rvqcw\" (UID: \"a6d54b5d-72a9-412c-8dbc-e4993832122a\") " pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:53.951497 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:53.951487    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:53.951599 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:53.951585    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret podName:a6d54b5d-72a9-412c-8dbc-e4993832122a nodeName:}" failed. No retries permitted until 2026-07-29 10:28:57.951565512 +0000 UTC m=+9.134921911 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret") pod "global-pull-secret-syncer-rvqcw" (UID: "a6d54b5d-72a9-412c-8dbc-e4993832122a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:53.952136 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:53.952113    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:53.952250 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:53.952170    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs podName:d598bd41-0d98-42fb-b082-8582a3b3b5ef nodeName:}" failed. No retries permitted until 2026-07-29 10:28:57.952155046 +0000 UTC m=+9.135511443 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs") pod "network-metrics-daemon-x5nh2" (UID: "d598bd41-0d98-42fb-b082-8582a3b3b5ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:54.052651 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:54.052613    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p4vp\" (UniqueName: \"kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp\") pod \"network-check-target-fp5xd\" (UID: \"33f96a45-1e06-4af4-bcbe-7e494cfcbfe2\") " pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:28:54.052858 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:54.052840    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 10:28:54.052942 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:54.052865    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 10:28:54.052942 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:54.052878    2564 projected.go:196] Error preparing data for projected volume kube-api-access-7p4vp for pod openshift-network-diagnostics/network-check-target-fp5xd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:54.053048 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:54.052943    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp podName:33f96a45-1e06-4af4-bcbe-7e494cfcbfe2 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:58.052924207 +0000 UTC m=+9.236280602 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7p4vp" (UniqueName: "kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp") pod "network-check-target-fp5xd" (UID: "33f96a45-1e06-4af4-bcbe-7e494cfcbfe2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:54.337851 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:54.337759    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:28:54.338014 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:54.337765    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:28:54.338014 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:54.337923    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:28:54.338112 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:54.338016    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:28:55.339622 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:55.338866    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:55.339622 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:55.338967    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:28:56.338122 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:56.338086    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:28:56.338289 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:56.338088    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:28:56.338289 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:56.338244    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:28:56.338409 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:56.338309    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:28:57.338538 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:57.338481    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:57.338994 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:57.338621    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:28:57.985610 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:57.985467    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs\") pod \"network-metrics-daemon-x5nh2\" (UID: \"d598bd41-0d98-42fb-b082-8582a3b3b5ef\") " pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:28:57.985610 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:57.985531    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret\") pod \"global-pull-secret-syncer-rvqcw\" (UID: \"a6d54b5d-72a9-412c-8dbc-e4993832122a\") " pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:57.985861 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:57.985622    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:57.985861 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:57.985671    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:57.985861 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:57.985700    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs podName:d598bd41-0d98-42fb-b082-8582a3b3b5ef nodeName:}" failed. No retries permitted until 2026-07-29 10:29:05.985679659 +0000 UTC m=+17.169036074 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs") pod "network-metrics-daemon-x5nh2" (UID: "d598bd41-0d98-42fb-b082-8582a3b3b5ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:57.985861 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:57.985738    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret podName:a6d54b5d-72a9-412c-8dbc-e4993832122a nodeName:}" failed. No retries permitted until 2026-07-29 10:29:05.985719286 +0000 UTC m=+17.169075681 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret") pod "global-pull-secret-syncer-rvqcw" (UID: "a6d54b5d-72a9-412c-8dbc-e4993832122a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:58.086490 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:58.086457    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p4vp\" (UniqueName: \"kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp\") pod \"network-check-target-fp5xd\" (UID: \"33f96a45-1e06-4af4-bcbe-7e494cfcbfe2\") " pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:28:58.086659 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:58.086587    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 10:28:58.086659 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:58.086603    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 10:28:58.086659 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:58.086612    2564 projected.go:196] Error preparing data for projected volume kube-api-access-7p4vp for pod openshift-network-diagnostics/network-check-target-fp5xd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:58.086659 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:58.086655    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp podName:33f96a45-1e06-4af4-bcbe-7e494cfcbfe2 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:06.086642507 +0000 UTC m=+17.269998901 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7p4vp" (UniqueName: "kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp") pod "network-check-target-fp5xd" (UID: "33f96a45-1e06-4af4-bcbe-7e494cfcbfe2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:58.338687 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:58.338607    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:28:58.339203 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:58.338738    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:28:58.339203 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:58.338812    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:28:58.339203 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:58.338911    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:28:59.340830 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:28:59.339896    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:28:59.340830 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:28:59.340062    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:29:00.338716 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:00.338631    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:00.338716 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:00.338673    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:29:00.338973 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:00.338750    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:29:00.338973 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:00.338916    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:29:01.338376 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:01.338342    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:01.338880 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:01.338467    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:29:02.338476 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:02.338384    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:02.338932 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:02.338517    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:29:02.338932 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:02.338575    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:29:02.338932 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:02.338696    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:29:03.337874 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:03.337839    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:03.338030 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:03.337972    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:29:04.338324 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:04.338281    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:29:04.338324 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:04.338331    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:04.338817 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:04.338420    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:29:04.338817 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:04.338547    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:29:05.337957 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:05.337918    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:05.338140 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:05.338055    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:29:06.050105 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:06.050074    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs\") pod \"network-metrics-daemon-x5nh2\" (UID: \"d598bd41-0d98-42fb-b082-8582a3b3b5ef\") " pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:29:06.050577 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:06.050135    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret\") pod \"global-pull-secret-syncer-rvqcw\" (UID: \"a6d54b5d-72a9-412c-8dbc-e4993832122a\") " pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:06.050577 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:06.050251    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:29:06.050577 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:06.050263    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 10:29:06.050577 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:06.050330    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs podName:d598bd41-0d98-42fb-b082-8582a3b3b5ef nodeName:}" failed. No retries permitted until 2026-07-29 10:29:22.050306331 +0000 UTC m=+33.233662727 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs") pod "network-metrics-daemon-x5nh2" (UID: "d598bd41-0d98-42fb-b082-8582a3b3b5ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:29:06.050577 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:06.050352    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret podName:a6d54b5d-72a9-412c-8dbc-e4993832122a nodeName:}" failed. No retries permitted until 2026-07-29 10:29:22.050341496 +0000 UTC m=+33.233697889 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret") pod "global-pull-secret-syncer-rvqcw" (UID: "a6d54b5d-72a9-412c-8dbc-e4993832122a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 10:29:06.150706 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:06.150668    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p4vp\" (UniqueName: \"kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp\") pod \"network-check-target-fp5xd\" (UID: \"33f96a45-1e06-4af4-bcbe-7e494cfcbfe2\") " pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:06.150893 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:06.150793    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 10:29:06.150893 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:06.150828    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 10:29:06.150893 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:06.150838    2564 projected.go:196] Error preparing data for projected volume kube-api-access-7p4vp for pod openshift-network-diagnostics/network-check-target-fp5xd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:29:06.150893 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:06.150893    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp podName:33f96a45-1e06-4af4-bcbe-7e494cfcbfe2 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:22.150876293 +0000 UTC m=+33.334232706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7p4vp" (UniqueName: "kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp") pod "network-check-target-fp5xd" (UID: "33f96a45-1e06-4af4-bcbe-7e494cfcbfe2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:29:06.338006 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:06.337918    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:06.338168 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:06.337927    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:29:06.338168 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:06.338044    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:29:06.338168 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:06.338135    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:29:07.338701 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:07.338667    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:07.339117 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:07.338820    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:29:08.338134 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:08.337931    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:08.338297 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:08.337934    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:29:08.338297 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:08.338244    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:29:08.338404 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:08.338315    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:29:09.338528 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:09.338490    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:09.338942 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:09.338608    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:29:10.338205 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:10.338178    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:10.338345 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:10.338307    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:29:10.339262 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:10.338663    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:29:10.339262 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:10.338766    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:29:10.451655 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:10.451616    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-233.ec2.internal" event={"ID":"ee950c7e37cd6646704499e83e8cb37d","Type":"ContainerStarted","Data":"3578d1ca5f42c7bb1ac4c21074c606d3ed9d93fb2ce4805ff861b2428d1a8741"}
Jul 29 10:29:10.453326 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:10.453301    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9jjlf" event={"ID":"0ad2e5c5-ef58-4346-90c9-6abcab412125","Type":"ContainerStarted","Data":"abc70855ca7f33f5abae10fcd47a5a7681b143b2c5eb9a697c4c99e0e0479b21"}
Jul 29 10:29:10.454701 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:10.454681    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mtx8j" event={"ID":"985d7ab1-89e7-4f96-908f-559b300397ea","Type":"ContainerStarted","Data":"3459325ac6598b0685e841c618fd0a5ce00543f543ac2250521b23350ee41c69"}
Jul 29 10:29:10.465279 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:10.465240    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-233.ec2.internal" podStartSLOduration=20.465228895 podStartE2EDuration="20.465228895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:10.465023476 +0000 UTC m=+21.648379891" watchObservedRunningTime="2026-07-29 10:29:10.465228895 +0000 UTC m=+21.648585310"
Jul 29 10:29:10.486024 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:10.485979    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mtx8j" podStartSLOduration=2.054187635 podStartE2EDuration="21.485965336s" totalImagesPullingTime="19.431777701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:10.485661935 +0000 UTC m=+21.669018349" watchObservedRunningTime="2026-07-29 10:29:10.485965336 +0000 UTC m=+21.669321751"
Jul 29 10:29:10.504639 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:10.504583    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9jjlf" podStartSLOduration=2.087813009 podStartE2EDuration="21.504562636s" totalImagesPullingTime="19.416749627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:10.504291225 +0000 UTC m=+21.687647642" watchObservedRunningTime="2026-07-29 10:29:10.504562636 +0000 UTC m=+21.687919053"
Jul 29 10:29:11.338551 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.338152    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:11.338693 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:11.338603    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:29:11.458975 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.458942    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wgrdq" event={"ID":"7d9c53a1-f5af-4bfc-81ba-4703f2fc039b","Type":"ContainerStarted","Data":"0691b59f4b2eb42813c8beae9e93da7cf804900132eb85bb7491cc8b8ed91889"}
Jul 29 10:29:11.460539 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.460514    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4" event={"ID":"5540a8cd-8b5e-496c-a681-d725efc6aa87","Type":"ContainerStarted","Data":"5e19063fc3b843b1562a5f871fef5a76dda9d36eb0715bdc091bb803149e2095"}
Jul 29 10:29:11.463771 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.463742    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf" event={"ID":"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7","Type":"ContainerStarted","Data":"be8c467d520c7d655d743886ad1d14dde15aa9378292b0e254a06f2babdf1709"}
Jul 29 10:29:11.463886 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.463826    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf" event={"ID":"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7","Type":"ContainerStarted","Data":"24a8b6691f914a696691f605f389de59de173fcd5ece94f2986dc0d935e371b8"}
Jul 29 10:29:11.463886 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.463862    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf" event={"ID":"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7","Type":"ContainerStarted","Data":"21cdc7bac05e9539b61434e3eec26d6b4dd2019377c4767fc886e3e7677ef904"}
Jul 29 10:29:11.464027 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.463904    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf" event={"ID":"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7","Type":"ContainerStarted","Data":"e860a2c9e93d4e32fdb25ea9fcff0387446c26e40550a285a39e673533ecf0bd"}
Jul 29 10:29:11.464027 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.463948    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf" event={"ID":"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7","Type":"ContainerStarted","Data":"d801c9e1d99309ce593049b5eee01d20dd7cb2934a60e9b7caca3cc4aec86640"}
Jul 29 10:29:11.464027 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.463991    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf" event={"ID":"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7","Type":"ContainerStarted","Data":"b8b56f0ecdf7d5adb8b87c1eb57b43aab86afe44f943f869eb987e3660893963"}
Jul 29 10:29:11.465382 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.465358    2564 generic.go:356] "Generic (PLEG): container finished" podID="dbf16cf7-3ffb-4dd6-a559-0d47817db5cf" containerID="b80256e822764e9958719a6015a8ab295c4c4c3b2a05fa74faa4e148c0297289" exitCode=0
Jul 29 10:29:11.465498 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.465444    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjgvn" event={"ID":"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf","Type":"ContainerDied","Data":"b80256e822764e9958719a6015a8ab295c4c4c3b2a05fa74faa4e148c0297289"}
Jul 29 10:29:11.467353 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.467330    2564 generic.go:356] "Generic (PLEG): container finished" podID="49addc5258e12ab015218ff440e6948d" containerID="7a3d8037e40e136b0553b0f9765a79802c278e0686db579b872f9baf602a8eed" exitCode=0
Jul 29 10:29:11.467458 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.467404    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal" event={"ID":"49addc5258e12ab015218ff440e6948d","Type":"ContainerDied","Data":"7a3d8037e40e136b0553b0f9765a79802c278e0686db579b872f9baf602a8eed"}
Jul 29 10:29:11.469058 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.469032    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j5xql" event={"ID":"3ffc02ea-68ae-4d60-82a4-01d159cd668c","Type":"ContainerStarted","Data":"6f4de43b66f4854c1c7237061acc8521b9c2954fcf9cda559707426d024c86ce"}
Jul 29 10:29:11.470627 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.470605    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dclkn" event={"ID":"9a04cac1-c3f9-42db-9339-54caac308ad2","Type":"ContainerStarted","Data":"601a4a1850b7d141aa42b5593d9ed0d19d43968203b6a89ae6b4f30a2742a86c"}
Jul 29 10:29:11.475149 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.475103    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wgrdq" podStartSLOduration=3.035633685 podStartE2EDuration="22.47508663s" totalImagesPullingTime="19.439452945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:11.474640738 +0000 UTC m=+22.657997156" watchObservedRunningTime="2026-07-29 10:29:11.47508663 +0000 UTC m=+22.658443045"
Jul 29 10:29:11.492007 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.491960    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-j5xql" podStartSLOduration=3.105247699 podStartE2EDuration="22.491945587s" totalImagesPullingTime="19.386697888s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:11.491287426 +0000 UTC m=+22.674643841" watchObservedRunningTime="2026-07-29 10:29:11.491945587 +0000 UTC m=+22.675302001"
Jul 29 10:29:11.529527 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.529469    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dclkn" podStartSLOduration=3.161903556 podStartE2EDuration="22.529449394s" totalImagesPullingTime="19.367545838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:11.528777609 +0000 UTC m=+22.712134052" watchObservedRunningTime="2026-07-29 10:29:11.529449394 +0000 UTC m=+22.712805811"
Jul 29 10:29:11.565862 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.565835    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dclkn"
Jul 29 10:29:11.566396 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:11.566380    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dclkn"
Jul 29 10:29:12.010742 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:12.010716    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 10:29:12.275618 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:12.275473    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T10:29:12.010734092Z","UUID":"ad9de588-5738-4c55-ac30-a09de5f09b12","Handler":null,"Name":"","Endpoint":""}
Jul 29 10:29:12.277179 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:12.277156    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 10:29:12.277308 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:12.277198    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 10:29:12.338612 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:12.338577    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:12.338762 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:12.338588    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:29:12.338762 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:12.338704    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:29:12.338905 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:12.338788    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:29:12.475434 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:12.475390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4" event={"ID":"5540a8cd-8b5e-496c-a681-d725efc6aa87","Type":"ContainerStarted","Data":"2b200450b559cf31d6ce0f1648959b3a7f31e91ce285da51e4c355317ebd8c84"}
Jul 29 10:29:12.477991 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:12.477534    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2mz7m" event={"ID":"c8eca8a0-11a6-488c-8856-53c63d71c5bf","Type":"ContainerStarted","Data":"9d4f4c3bffa4dc219b53033d53acdc637a066dd2d81a7039b077090bd6ecf4d8"}
Jul 29 10:29:12.480373 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:12.480335    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal" event={"ID":"49addc5258e12ab015218ff440e6948d","Type":"ContainerStarted","Data":"7fd5e771070da07a111afd922878a5ea47e445be3f0b67a33fc73ee1d4bdbcee"}
Jul 29 10:29:12.480531 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:12.480515    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dclkn"
Jul 29 10:29:12.481677 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:12.481657    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dclkn"
Jul 29 10:29:12.496520 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:12.496469    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2mz7m" podStartSLOduration=4.041272963 podStartE2EDuration="23.49645368s" totalImagesPullingTime="19.455180717s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:12.496397349 +0000 UTC m=+23.679753789" watchObservedRunningTime="2026-07-29 10:29:12.49645368 +0000 UTC m=+23.679810101"
Jul 29 10:29:12.512714 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:12.512663    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-233.ec2.internal" podStartSLOduration=22.512644386 podStartE2EDuration="22.512644386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:12.51236651 +0000 UTC m=+23.695722925" watchObservedRunningTime="2026-07-29 10:29:12.512644386 +0000 UTC m=+23.696000804"
Jul 29 10:29:13.338696 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:13.338669    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:13.338850 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:13.338776    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:29:13.484767 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:13.484676    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4" event={"ID":"5540a8cd-8b5e-496c-a681-d725efc6aa87","Type":"ContainerStarted","Data":"8dec99445b56cf16682315225b6628635bc197d83fa4478691aa0477b4821304"}
Jul 29 10:29:13.488438 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:13.488406    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf" event={"ID":"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7","Type":"ContainerStarted","Data":"275d7f527fb1676cc2bd7c724da7b0ffd76a2638a823590649f3b013e70e0135"}
Jul 29 10:29:13.505692 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:13.505649    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t9z4" podStartSLOduration=2.8425027099999998 podStartE2EDuration="24.505635958s" totalImagesPullingTime="21.663133248s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:13.50546514 +0000 UTC m=+24.688821574" watchObservedRunningTime="2026-07-29 10:29:13.505635958 +0000 UTC m=+24.688992373"
Jul 29 10:29:14.338171 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:14.338135    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:14.338335 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:14.338183    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:29:14.338335 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:14.338265    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:29:14.338423 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:14.338370    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:29:15.338130 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:15.338081    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:15.338576 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:15.338237    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:29:16.338553 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:16.338385    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:16.339200 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:16.338444    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:29:16.339200 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:16.338614    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:29:16.339200 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:16.338717    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:29:16.499499 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:16.498918    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf" event={"ID":"1ab4b50a-79d2-43f1-b4c0-e7459adb20c7","Type":"ContainerStarted","Data":"ec2978803d5036f2a831f1137fcf19e79c3790dd19a7e9b183e466284627a0e2"}
Jul 29 10:29:16.499711 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:16.499601    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:29:16.499711 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:16.499658    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:29:16.499711 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:16.499704    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:29:16.501422 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:16.501397    2564 generic.go:356] "Generic (PLEG): container finished" podID="dbf16cf7-3ffb-4dd6-a559-0d47817db5cf" containerID="07cc78c0ac18f5a4002ed239cf0ceb48fd126bc9741a845e457cb93eefa91dc0" exitCode=0
Jul 29 10:29:16.501530 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:16.501446    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjgvn" event={"ID":"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf","Type":"ContainerDied","Data":"07cc78c0ac18f5a4002ed239cf0ceb48fd126bc9741a845e457cb93eefa91dc0"}
Jul 29 10:29:16.515595 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:16.515576    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:29:16.515693 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:16.515667    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:29:16.526138 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:16.526086    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf" podStartSLOduration=7.588749054 podStartE2EDuration="27.526072969s" totalImagesPullingTime="19.937323915s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:16.524417874 +0000 UTC m=+27.707774288" watchObservedRunningTime="2026-07-29 10:29:16.526072969 +0000 UTC m=+27.709429383"
Jul 29 10:29:17.338461 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:17.338427    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:17.338722 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:17.338572    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:29:17.839715 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:17.839530    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fp5xd"]
Jul 29 10:29:17.839900 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:17.839861    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:17.839993 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:17.839972    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:29:17.842393 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:17.842371    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rvqcw"]
Jul 29 10:29:17.842486 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:17.842449    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:17.842553 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:17.842528    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:29:17.843165 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:17.843143    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x5nh2"]
Jul 29 10:29:17.843253 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:17.843227    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:29:17.843323 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:17.843308    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:29:18.507592 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:18.507557    2564 generic.go:356] "Generic (PLEG): container finished" podID="dbf16cf7-3ffb-4dd6-a559-0d47817db5cf" containerID="96cdcacd4e4a3e724864705a1ea4afce72d3c7b45584ffb299e354944ccc422e" exitCode=0
Jul 29 10:29:18.508093 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:18.507639    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjgvn" event={"ID":"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf","Type":"ContainerDied","Data":"96cdcacd4e4a3e724864705a1ea4afce72d3c7b45584ffb299e354944ccc422e"}
Jul 29 10:29:19.339025 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:19.338991    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:19.339183 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:19.339104    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:29:19.339223 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:19.339210    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:19.339351 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:19.339324    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:29:19.339386 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:19.339363    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:29:19.339439 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:19.339427    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:29:19.512612 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:19.512587    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjgvn" event={"ID":"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf","Type":"ContainerStarted","Data":"ab79b9962aa507ce39b9dcfda5175b57b20d55488d83ada7eeece6f4b9ee47fc"}
Jul 29 10:29:20.517097 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:20.517008    2564 generic.go:356] "Generic (PLEG): container finished" podID="dbf16cf7-3ffb-4dd6-a559-0d47817db5cf" containerID="ab79b9962aa507ce39b9dcfda5175b57b20d55488d83ada7eeece6f4b9ee47fc" exitCode=0
Jul 29 10:29:20.517097 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:20.517047    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjgvn" event={"ID":"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf","Type":"ContainerDied","Data":"ab79b9962aa507ce39b9dcfda5175b57b20d55488d83ada7eeece6f4b9ee47fc"}
Jul 29 10:29:21.338578 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:21.338543    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:21.338578 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:21.338561    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:29:21.338824 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:21.338660    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:21.338824 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:21.338678    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rvqcw" podUID="a6d54b5d-72a9-412c-8dbc-e4993832122a"
Jul 29 10:29:21.338824 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:21.338747    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp5xd" podUID="33f96a45-1e06-4af4-bcbe-7e494cfcbfe2"
Jul 29 10:29:21.338942 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:21.338877    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x5nh2" podUID="d598bd41-0d98-42fb-b082-8582a3b3b5ef"
Jul 29 10:29:22.073593 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:22.073561    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs\") pod \"network-metrics-daemon-x5nh2\" (UID: \"d598bd41-0d98-42fb-b082-8582a3b3b5ef\") " pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:29:22.074050 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:22.073629    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret\") pod \"global-pull-secret-syncer-rvqcw\" (UID: \"a6d54b5d-72a9-412c-8dbc-e4993832122a\") " pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:22.074050 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:22.073742    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 10:29:22.074050 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:22.073744    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:29:22.074050 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:22.073831    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret podName:a6d54b5d-72a9-412c-8dbc-e4993832122a nodeName:}" failed. No retries permitted until 2026-07-29 10:29:54.073795434 +0000 UTC m=+65.257151839 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret") pod "global-pull-secret-syncer-rvqcw" (UID: "a6d54b5d-72a9-412c-8dbc-e4993832122a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 10:29:22.074050 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:22.073851    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs podName:d598bd41-0d98-42fb-b082-8582a3b3b5ef nodeName:}" failed. No retries permitted until 2026-07-29 10:29:54.073841143 +0000 UTC m=+65.257197551 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs") pod "network-metrics-daemon-x5nh2" (UID: "d598bd41-0d98-42fb-b082-8582a3b3b5ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:29:22.174717 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:22.174681    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p4vp\" (UniqueName: \"kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp\") pod \"network-check-target-fp5xd\" (UID: \"33f96a45-1e06-4af4-bcbe-7e494cfcbfe2\") " pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:22.174915 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:22.174862    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 10:29:22.174915 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:22.174886    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 10:29:22.174915 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:22.174896    2564 projected.go:196] Error preparing data for projected volume kube-api-access-7p4vp for pod openshift-network-diagnostics/network-check-target-fp5xd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:29:22.175075 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:22.174949    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp podName:33f96a45-1e06-4af4-bcbe-7e494cfcbfe2 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:54.174931613 +0000 UTC m=+65.358288026 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7p4vp" (UniqueName: "kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp") pod "network-check-target-fp5xd" (UID: "33f96a45-1e06-4af4-bcbe-7e494cfcbfe2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:29:23.131956 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.131925    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-233.ec2.internal" event="NodeReady"
Jul 29 10:29:23.132398 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.132070    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 10:29:23.183609 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.183521    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hhbsk"]
Jul 29 10:29:23.185657 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.185630    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8dxdp"]
Jul 29 10:29:23.185783 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.185757    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:23.187558 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.187537    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8dxdp"
Jul 29 10:29:23.188312 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.188293    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hm7tz\""
Jul 29 10:29:23.188531 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.188510    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 10:29:23.188531 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.188514    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 10:29:23.190042 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.190020    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 10:29:23.190220 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.190202    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 10:29:23.190310 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.190280    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9bsc6\""
Jul 29 10:29:23.190518 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.190492    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 10:29:23.195555 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.195534    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8dxdp"]
Jul 29 10:29:23.199168 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.199148    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hhbsk"]
Jul 29 10:29:23.285295 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.285266    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73102ac7-c5cf-43b4-a289-4976d3e902e9-config-volume\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:23.285451 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.285299    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73102ac7-c5cf-43b4-a289-4976d3e902e9-tmp-dir\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:23.285451 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.285326    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:23.285451 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.285383    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g4x9\" (UniqueName: \"kubernetes.io/projected/04cbd6e9-e56f-4e33-bc1a-901eeec3248d-kube-api-access-5g4x9\") pod \"ingress-canary-8dxdp\" (UID: \"04cbd6e9-e56f-4e33-bc1a-901eeec3248d\") " pod="openshift-ingress-canary/ingress-canary-8dxdp"
Jul 29 10:29:23.285579 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.285452    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04cbd6e9-e56f-4e33-bc1a-901eeec3248d-cert\") pod \"ingress-canary-8dxdp\" (UID: \"04cbd6e9-e56f-4e33-bc1a-901eeec3248d\") " pod="openshift-ingress-canary/ingress-canary-8dxdp"
Jul 29 10:29:23.285579 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.285476    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzqnx\" (UniqueName: \"kubernetes.io/projected/73102ac7-c5cf-43b4-a289-4976d3e902e9-kube-api-access-mzqnx\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:23.338611 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.338583    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:29:23.338756 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.338724    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:23.338978 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.338925    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:23.343086 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.343064    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 10:29:23.343207 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.343141    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 10:29:23.343207 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.343171    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n9sqp\""
Jul 29 10:29:23.343207 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.343183    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 10:29:23.343382 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.343363    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2mpdj\""
Jul 29 10:29:23.343609 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.343593    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 10:29:23.386023 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.385994    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzqnx\" (UniqueName: \"kubernetes.io/projected/73102ac7-c5cf-43b4-a289-4976d3e902e9-kube-api-access-mzqnx\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:23.386170 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.386075    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73102ac7-c5cf-43b4-a289-4976d3e902e9-config-volume\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:23.386170 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.386107    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73102ac7-c5cf-43b4-a289-4976d3e902e9-tmp-dir\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:23.386170 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.386132    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:23.386170 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.386166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g4x9\" (UniqueName: \"kubernetes.io/projected/04cbd6e9-e56f-4e33-bc1a-901eeec3248d-kube-api-access-5g4x9\") pod \"ingress-canary-8dxdp\" (UID: \"04cbd6e9-e56f-4e33-bc1a-901eeec3248d\") " pod="openshift-ingress-canary/ingress-canary-8dxdp"
Jul 29 10:29:23.386332 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.386216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04cbd6e9-e56f-4e33-bc1a-901eeec3248d-cert\") pod \"ingress-canary-8dxdp\" (UID: \"04cbd6e9-e56f-4e33-bc1a-901eeec3248d\") " pod="openshift-ingress-canary/ingress-canary-8dxdp"
Jul 29 10:29:23.386332 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:23.386266    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 10:29:23.386394 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:23.386345    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls podName:73102ac7-c5cf-43b4-a289-4976d3e902e9 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:23.886314079 +0000 UTC m=+35.069670486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls") pod "dns-default-hhbsk" (UID: "73102ac7-c5cf-43b4-a289-4976d3e902e9") : secret "dns-default-metrics-tls" not found
Jul 29 10:29:23.386573 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.386546    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73102ac7-c5cf-43b4-a289-4976d3e902e9-tmp-dir\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:23.387111 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.387089    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73102ac7-c5cf-43b4-a289-4976d3e902e9-config-volume\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:23.391641 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.391502    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04cbd6e9-e56f-4e33-bc1a-901eeec3248d-cert\") pod \"ingress-canary-8dxdp\" (UID: \"04cbd6e9-e56f-4e33-bc1a-901eeec3248d\") " pod="openshift-ingress-canary/ingress-canary-8dxdp"
Jul 29 10:29:23.395531 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.395480    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g4x9\" (UniqueName: \"kubernetes.io/projected/04cbd6e9-e56f-4e33-bc1a-901eeec3248d-kube-api-access-5g4x9\") pod \"ingress-canary-8dxdp\" (UID: \"04cbd6e9-e56f-4e33-bc1a-901eeec3248d\") " pod="openshift-ingress-canary/ingress-canary-8dxdp"
Jul 29 10:29:23.395531 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.395508    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzqnx\" (UniqueName: \"kubernetes.io/projected/73102ac7-c5cf-43b4-a289-4976d3e902e9-kube-api-access-mzqnx\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:23.505421 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.505390    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8dxdp"
Jul 29 10:29:23.891079 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:23.891000    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:23.891237 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:23.891170    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 10:29:23.891286 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:23.891249    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls podName:73102ac7-c5cf-43b4-a289-4976d3e902e9 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:24.891227399 +0000 UTC m=+36.074583795 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls") pod "dns-default-hhbsk" (UID: "73102ac7-c5cf-43b4-a289-4976d3e902e9") : secret "dns-default-metrics-tls" not found
Jul 29 10:29:24.815898 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:24.815870    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8dxdp"]
Jul 29 10:29:24.819337 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:29:24.819307    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04cbd6e9_e56f_4e33_bc1a_901eeec3248d.slice/crio-f92bec982bf34a1a7c01b02f6ae1e5643048fa02c6c4521598168d285a59be6a WatchSource:0}: Error finding container f92bec982bf34a1a7c01b02f6ae1e5643048fa02c6c4521598168d285a59be6a: Status 404 returned error can't find the container with id f92bec982bf34a1a7c01b02f6ae1e5643048fa02c6c4521598168d285a59be6a
Jul 29 10:29:24.899866 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:24.899836    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:24.900029 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:24.900010    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 10:29:24.900093 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:24.900078    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls podName:73102ac7-c5cf-43b4-a289-4976d3e902e9 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:26.900062154 +0000 UTC m=+38.083418551 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls") pod "dns-default-hhbsk" (UID: "73102ac7-c5cf-43b4-a289-4976d3e902e9") : secret "dns-default-metrics-tls" not found
Jul 29 10:29:25.531891 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:25.531851    2564 generic.go:356] "Generic (PLEG): container finished" podID="dbf16cf7-3ffb-4dd6-a559-0d47817db5cf" containerID="47aae17a86f0a1b598cc0cb7acbddc5f53cd6a92def083920e6ad9d1fe95211b" exitCode=0
Jul 29 10:29:25.532130 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:25.531897    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjgvn" event={"ID":"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf","Type":"ContainerDied","Data":"47aae17a86f0a1b598cc0cb7acbddc5f53cd6a92def083920e6ad9d1fe95211b"}
Jul 29 10:29:25.533605 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:25.533570    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8dxdp" event={"ID":"04cbd6e9-e56f-4e33-bc1a-901eeec3248d","Type":"ContainerStarted","Data":"f92bec982bf34a1a7c01b02f6ae1e5643048fa02c6c4521598168d285a59be6a"}
Jul 29 10:29:26.539405 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:26.539367    2564 generic.go:356] "Generic (PLEG): container finished" podID="dbf16cf7-3ffb-4dd6-a559-0d47817db5cf" containerID="bbd7d77a0a12f2a45610aaee8c6c595f1f961346f61cb3506d2053eb2d52d533" exitCode=0
Jul 29 10:29:26.539872 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:26.539406    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjgvn" event={"ID":"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf","Type":"ContainerDied","Data":"bbd7d77a0a12f2a45610aaee8c6c595f1f961346f61cb3506d2053eb2d52d533"}
Jul 29 10:29:26.916845 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:26.916823    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:26.917015 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:26.916995    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 10:29:26.917096 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:26.917084    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls podName:73102ac7-c5cf-43b4-a289-4976d3e902e9 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:30.917062018 +0000 UTC m=+42.100418415 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls") pod "dns-default-hhbsk" (UID: "73102ac7-c5cf-43b4-a289-4976d3e902e9") : secret "dns-default-metrics-tls" not found
Jul 29 10:29:27.544978 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:27.544940    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjgvn" event={"ID":"dbf16cf7-3ffb-4dd6-a559-0d47817db5cf","Type":"ContainerStarted","Data":"086a90976a1dd44277bb753f7b23efc37f90d35c988e52015004b893d46853e6"}
Jul 29 10:29:27.546376 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:27.546350    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8dxdp" event={"ID":"04cbd6e9-e56f-4e33-bc1a-901eeec3248d","Type":"ContainerStarted","Data":"960ba49b08c1dd7ad5f8004271000d8fe1fabd42e9672d16d8ed931eb0d668b5"}
Jul 29 10:29:27.574784 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:27.574734    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rjgvn" podStartSLOduration=8.70996446 podStartE2EDuration="38.574718856s" totalImagesPullingTime="29.864754396s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:27.573040081 +0000 UTC m=+38.756396496" watchObservedRunningTime="2026-07-29 10:29:27.574718856 +0000 UTC m=+38.758075271"
Jul 29 10:29:27.596252 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:27.596202    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8dxdp" podStartSLOduration=2.207705585 podStartE2EDuration="4.596184474s" totalImagesPullingTime="2.388478889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:27.595737464 +0000 UTC m=+38.779093879" watchObservedRunningTime="2026-07-29 10:29:27.596184474 +0000 UTC m=+38.779541013"
Jul 29 10:29:30.946609 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:30.946566    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:30.947006 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:30.946713    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 10:29:30.947006 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:30.946772    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls podName:73102ac7-c5cf-43b4-a289-4976d3e902e9 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:38.946756113 +0000 UTC m=+50.130112509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls") pod "dns-default-hhbsk" (UID: "73102ac7-c5cf-43b4-a289-4976d3e902e9") : secret "dns-default-metrics-tls" not found
Jul 29 10:29:39.004776 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:39.004728    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:39.005263 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:39.004900    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 10:29:39.005263 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:39.004975    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls podName:73102ac7-c5cf-43b4-a289-4976d3e902e9 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:55.004958477 +0000 UTC m=+66.188314870 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls") pod "dns-default-hhbsk" (UID: "73102ac7-c5cf-43b4-a289-4976d3e902e9") : secret "dns-default-metrics-tls" not found
Jul 29 10:29:48.519009 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:48.518980    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-psdkf"
Jul 29 10:29:54.106085 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:54.106032    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret\") pod \"global-pull-secret-syncer-rvqcw\" (UID: \"a6d54b5d-72a9-412c-8dbc-e4993832122a\") " pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:54.106585 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:54.106123    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs\") pod \"network-metrics-daemon-x5nh2\" (UID: \"d598bd41-0d98-42fb-b082-8582a3b3b5ef\") " pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:29:54.108781 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:54.108758    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 10:29:54.108882 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:54.108817    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 10:29:54.117043 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:54.117024    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 29 10:29:54.117104 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:54.117081    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs podName:d598bd41-0d98-42fb-b082-8582a3b3b5ef nodeName:}" failed. No retries permitted until 2026-07-29 10:30:58.117064757 +0000 UTC m=+129.300421150 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs") pod "network-metrics-daemon-x5nh2" (UID: "d598bd41-0d98-42fb-b082-8582a3b3b5ef") : secret "metrics-daemon-secret" not found
Jul 29 10:29:54.120378 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:54.120354    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6d54b5d-72a9-412c-8dbc-e4993832122a-original-pull-secret\") pod \"global-pull-secret-syncer-rvqcw\" (UID: \"a6d54b5d-72a9-412c-8dbc-e4993832122a\") " pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:54.207228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:54.207191    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p4vp\" (UniqueName: \"kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp\") pod \"network-check-target-fp5xd\" (UID: \"33f96a45-1e06-4af4-bcbe-7e494cfcbfe2\") " pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:54.210516 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:54.210492    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 10:29:54.222524 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:54.222495    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 10:29:54.230453 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:54.230429    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p4vp\" (UniqueName: \"kubernetes.io/projected/33f96a45-1e06-4af4-bcbe-7e494cfcbfe2-kube-api-access-7p4vp\") pod \"network-check-target-fp5xd\" (UID: \"33f96a45-1e06-4af4-bcbe-7e494cfcbfe2\") " pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:54.256280 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:54.256252    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rvqcw"
Jul 29 10:29:54.264889 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:54.264871    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2mpdj\""
Jul 29 10:29:54.272644 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:54.272623    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:29:54.385957 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:54.385885    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rvqcw"]
Jul 29 10:29:54.388440 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:29:54.388415    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6d54b5d_72a9_412c_8dbc_e4993832122a.slice/crio-16209b5aa26952565de048a815452cf1974c160d0c4bba9c5fe98454b07d6347 WatchSource:0}: Error finding container 16209b5aa26952565de048a815452cf1974c160d0c4bba9c5fe98454b07d6347: Status 404 returned error can't find the container with id 16209b5aa26952565de048a815452cf1974c160d0c4bba9c5fe98454b07d6347
Jul 29 10:29:54.402466 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:54.402440    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fp5xd"]
Jul 29 10:29:54.405248 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:29:54.405226    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33f96a45_1e06_4af4_bcbe_7e494cfcbfe2.slice/crio-5bd6e2b1ac2b5d61b1ee26a09c3b45b650272c892caa8dc5e12528a727d48d64 WatchSource:0}: Error finding container 5bd6e2b1ac2b5d61b1ee26a09c3b45b650272c892caa8dc5e12528a727d48d64: Status 404 returned error can't find the container with id 5bd6e2b1ac2b5d61b1ee26a09c3b45b650272c892caa8dc5e12528a727d48d64
Jul 29 10:29:54.614040 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:54.614000    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rvqcw" event={"ID":"a6d54b5d-72a9-412c-8dbc-e4993832122a","Type":"ContainerStarted","Data":"16209b5aa26952565de048a815452cf1974c160d0c4bba9c5fe98454b07d6347"}
Jul 29 10:29:54.615015 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:54.614990    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fp5xd" event={"ID":"33f96a45-1e06-4af4-bcbe-7e494cfcbfe2","Type":"ContainerStarted","Data":"5bd6e2b1ac2b5d61b1ee26a09c3b45b650272c892caa8dc5e12528a727d48d64"}
Jul 29 10:29:55.013189 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.013149    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:29:55.013358 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:55.013296    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 10:29:55.013423 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:55.013373    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls podName:73102ac7-c5cf-43b4-a289-4976d3e902e9 nodeName:}" failed. No retries permitted until 2026-07-29 10:30:27.013352878 +0000 UTC m=+98.196709272 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls") pod "dns-default-hhbsk" (UID: "73102ac7-c5cf-43b4-a289-4976d3e902e9") : secret "dns-default-metrics-tls" not found
Jul 29 10:29:55.094720 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.094680    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fh4qq"]
Jul 29 10:29:55.097557 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.096640    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5"]
Jul 29 10:29:55.097557 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.096731    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fh4qq"
Jul 29 10:29:55.098610 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.098587    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5"
Jul 29 10:29:55.099735 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.099552    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 29 10:29:55.099919 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.099874    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 29 10:29:55.100255 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.100235    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8c4kz\""
Jul 29 10:29:55.101298 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.101278    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 29 10:29:55.102728 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.101861    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-sl8dz\""
Jul 29 10:29:55.102728 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.102108    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 29 10:29:55.102728 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.102463    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 29 10:29:55.105004 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.104922    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 29 10:29:55.109875 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.109823    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5"]
Jul 29 10:29:55.113386 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.113326    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fh4qq"]
Jul 29 10:29:55.113874 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.113639    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22f26925-6582-4316-a1b1-a451fe0ecba8-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5\" (UID: \"22f26925-6582-4316-a1b1-a451fe0ecba8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5"
Jul 29 10:29:55.113874 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.113685    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmqbg\" (UniqueName: \"kubernetes.io/projected/22f26925-6582-4316-a1b1-a451fe0ecba8-kube-api-access-hmqbg\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5\" (UID: \"22f26925-6582-4316-a1b1-a451fe0ecba8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5"
Jul 29 10:29:55.113874 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.113764    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4hp5\" (UniqueName: \"kubernetes.io/projected/e6ba5cdf-2ce5-4286-8a74-6f9f0b89e75d-kube-api-access-l4hp5\") pod \"volume-data-source-validator-549c6d54cf-fh4qq\" (UID: \"e6ba5cdf-2ce5-4286-8a74-6f9f0b89e75d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fh4qq"
Jul 29 10:29:55.113874 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.113822    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22f26925-6582-4316-a1b1-a451fe0ecba8-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5\" (UID: \"22f26925-6582-4316-a1b1-a451fe0ecba8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5"
Jul 29 10:29:55.186688 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.186655    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64ff54b9f5-xd2bg"]
Jul 29 10:29:55.188936 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.188858    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.193037 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.192884    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 10:29:55.193210 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.193190    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c2kld\""
Jul 29 10:29:55.193319 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.193293    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 10:29:55.194162 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.194139    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 10:29:55.199170 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.198984    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 10:29:55.206386 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.206351    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64ff54b9f5-xd2bg"]
Jul 29 10:29:55.214822 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.214578    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22f26925-6582-4316-a1b1-a451fe0ecba8-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5\" (UID: \"22f26925-6582-4316-a1b1-a451fe0ecba8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5"
Jul 29 10:29:55.214822 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.214620    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.214822 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.214639    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f028e32c-2c95-43b6-907e-79916611f42f-registry-certificates\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.214822 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.214655    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-bound-sa-token\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.214822 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.214669    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhcn7\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-kube-api-access-vhcn7\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.214822 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.214694    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmqbg\" (UniqueName: \"kubernetes.io/projected/22f26925-6582-4316-a1b1-a451fe0ecba8-kube-api-access-hmqbg\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5\" (UID: \"22f26925-6582-4316-a1b1-a451fe0ecba8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5"
Jul 29 10:29:55.214822 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.214730    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f028e32c-2c95-43b6-907e-79916611f42f-image-registry-private-configuration\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.214822 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.214750    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4hp5\" (UniqueName: \"kubernetes.io/projected/e6ba5cdf-2ce5-4286-8a74-6f9f0b89e75d-kube-api-access-l4hp5\") pod \"volume-data-source-validator-549c6d54cf-fh4qq\" (UID: \"e6ba5cdf-2ce5-4286-8a74-6f9f0b89e75d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fh4qq"
Jul 29 10:29:55.214822 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.214774    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f028e32c-2c95-43b6-907e-79916611f42f-ca-trust-extracted\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.215396 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.214844    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f028e32c-2c95-43b6-907e-79916611f42f-installation-pull-secrets\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.215396 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.214876    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22f26925-6582-4316-a1b1-a451fe0ecba8-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5\" (UID: \"22f26925-6582-4316-a1b1-a451fe0ecba8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5"
Jul 29 10:29:55.215396 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.214943    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f028e32c-2c95-43b6-907e-79916611f42f-trusted-ca\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.215396 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.215304    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22f26925-6582-4316-a1b1-a451fe0ecba8-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5\" (UID: \"22f26925-6582-4316-a1b1-a451fe0ecba8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5"
Jul 29 10:29:55.217564 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.217542    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22f26925-6582-4316-a1b1-a451fe0ecba8-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5\" (UID: \"22f26925-6582-4316-a1b1-a451fe0ecba8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5"
Jul 29 10:29:55.227598 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.227528    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmqbg\" (UniqueName: \"kubernetes.io/projected/22f26925-6582-4316-a1b1-a451fe0ecba8-kube-api-access-hmqbg\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5\" (UID: \"22f26925-6582-4316-a1b1-a451fe0ecba8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5"
Jul 29 10:29:55.227598 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.227577    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4hp5\" (UniqueName: \"kubernetes.io/projected/e6ba5cdf-2ce5-4286-8a74-6f9f0b89e75d-kube-api-access-l4hp5\") pod \"volume-data-source-validator-549c6d54cf-fh4qq\" (UID: \"e6ba5cdf-2ce5-4286-8a74-6f9f0b89e75d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fh4qq"
Jul 29 10:29:55.289244 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.289118    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-vm85k"]
Jul 29 10:29:55.291233 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.291208    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.293831 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.293786    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 29 10:29:55.296027 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.296001    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 10:29:55.296249 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.296231    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-zh44w\""
Jul 29 10:29:55.296611 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.296442    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf"]
Jul 29 10:29:55.296611 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.296532    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 10:29:55.297237 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.297219    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 29 10:29:55.298854 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.298473    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf"
Jul 29 10:29:55.305999 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.305978    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 29 10:29:55.306118 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.306098    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 29 10:29:55.306118 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.306108    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 29 10:29:55.306658 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.306637    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-94jbt\""
Jul 29 10:29:55.315118 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.314933    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-vm85k"]
Jul 29 10:29:55.315233 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.315217    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4875bd25-9d39-432d-bffa-2f422607180d-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.315298 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.315255    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hlxcf\" (UID: \"6f242b84-732d-4642-b7a9-8e6c874d0bdd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf"
Jul 29 10:29:55.315298 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.315280    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hffth\" (UniqueName: \"kubernetes.io/projected/6f242b84-732d-4642-b7a9-8e6c874d0bdd-kube-api-access-hffth\") pod \"cluster-samples-operator-6748489656-hlxcf\" (UID: \"6f242b84-732d-4642-b7a9-8e6c874d0bdd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf"
Jul 29 10:29:55.315394 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.315316    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f028e32c-2c95-43b6-907e-79916611f42f-trusted-ca\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.315394 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.315340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4875bd25-9d39-432d-bffa-2f422607180d-tmp\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.315394 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.315364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4875bd25-9d39-432d-bffa-2f422607180d-serving-cert\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.315526 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.315391    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnt5g\" (UniqueName: \"kubernetes.io/projected/4875bd25-9d39-432d-bffa-2f422607180d-kube-api-access-fnt5g\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.315526 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.315426    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.315526 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.315449    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f028e32c-2c95-43b6-907e-79916611f42f-registry-certificates\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.315526 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.315476    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-bound-sa-token\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.315526 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.315501    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhcn7\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-kube-api-access-vhcn7\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.315877 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.315524    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4875bd25-9d39-432d-bffa-2f422607180d-snapshots\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.315877 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.315549    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4875bd25-9d39-432d-bffa-2f422607180d-service-ca-bundle\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.315877 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.315607    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f028e32c-2c95-43b6-907e-79916611f42f-image-registry-private-configuration\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.315877 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.315637    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f028e32c-2c95-43b6-907e-79916611f42f-ca-trust-extracted\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.315877 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.315661    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f028e32c-2c95-43b6-907e-79916611f42f-installation-pull-secrets\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.317167 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:55.316482    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 10:29:55.317167 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:55.316501    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64ff54b9f5-xd2bg: secret "image-registry-tls" not found
Jul 29 10:29:55.317167 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:55.316551    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls podName:f028e32c-2c95-43b6-907e-79916611f42f nodeName:}" failed. No retries permitted until 2026-07-29 10:29:55.816533721 +0000 UTC m=+66.999890143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls") pod "image-registry-64ff54b9f5-xd2bg" (UID: "f028e32c-2c95-43b6-907e-79916611f42f") : secret "image-registry-tls" not found
Jul 29 10:29:55.317167 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.316589    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f028e32c-2c95-43b6-907e-79916611f42f-trusted-ca\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.317167 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.317112    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f028e32c-2c95-43b6-907e-79916611f42f-registry-certificates\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.317589 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.317437    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f028e32c-2c95-43b6-907e-79916611f42f-ca-trust-extracted\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.320154 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.318605    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f028e32c-2c95-43b6-907e-79916611f42f-installation-pull-secrets\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.320154 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.319241    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 29 10:29:55.320154 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.320075    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f028e32c-2c95-43b6-907e-79916611f42f-image-registry-private-configuration\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.325678 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.325646    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf"]
Jul 29 10:29:55.340582 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.340553    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-bound-sa-token\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.345223 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.345202    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhcn7\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-kube-api-access-vhcn7\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.410971 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.410934    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fh4qq"
Jul 29 10:29:55.416985 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.416943    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4875bd25-9d39-432d-bffa-2f422607180d-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.417119 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.416993    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hlxcf\" (UID: \"6f242b84-732d-4642-b7a9-8e6c874d0bdd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf"
Jul 29 10:29:55.417119 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.417047    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hffth\" (UniqueName: \"kubernetes.io/projected/6f242b84-732d-4642-b7a9-8e6c874d0bdd-kube-api-access-hffth\") pod \"cluster-samples-operator-6748489656-hlxcf\" (UID: \"6f242b84-732d-4642-b7a9-8e6c874d0bdd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf"
Jul 29 10:29:55.417119 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.417068    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4875bd25-9d39-432d-bffa-2f422607180d-tmp\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.417119 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.417087    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4875bd25-9d39-432d-bffa-2f422607180d-serving-cert\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.417119 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.417115    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnt5g\" (UniqueName: \"kubernetes.io/projected/4875bd25-9d39-432d-bffa-2f422607180d-kube-api-access-fnt5g\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.417381 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.417165    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4875bd25-9d39-432d-bffa-2f422607180d-snapshots\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.417381 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:55.417177    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 10:29:55.417381 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:55.417251    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls podName:6f242b84-732d-4642-b7a9-8e6c874d0bdd nodeName:}" failed. No retries permitted until 2026-07-29 10:29:55.91722917 +0000 UTC m=+67.100585563 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls") pod "cluster-samples-operator-6748489656-hlxcf" (UID: "6f242b84-732d-4642-b7a9-8e6c874d0bdd") : secret "samples-operator-tls" not found
Jul 29 10:29:55.417381 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.417187    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4875bd25-9d39-432d-bffa-2f422607180d-service-ca-bundle\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.417749 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.417718    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4875bd25-9d39-432d-bffa-2f422607180d-service-ca-bundle\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.417947 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.417929    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4875bd25-9d39-432d-bffa-2f422607180d-tmp\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.418035 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.417942    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4875bd25-9d39-432d-bffa-2f422607180d-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.418284 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.418227    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4875bd25-9d39-432d-bffa-2f422607180d-snapshots\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.420119 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.419954    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5"
Jul 29 10:29:55.420243 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.420221    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4875bd25-9d39-432d-bffa-2f422607180d-serving-cert\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.430724 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.430695    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnt5g\" (UniqueName: \"kubernetes.io/projected/4875bd25-9d39-432d-bffa-2f422607180d-kube-api-access-fnt5g\") pod \"insights-operator-5885fdb759-vm85k\" (UID: \"4875bd25-9d39-432d-bffa-2f422607180d\") " pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.432379 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.432168    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hffth\" (UniqueName: \"kubernetes.io/projected/6f242b84-732d-4642-b7a9-8e6c874d0bdd-kube-api-access-hffth\") pod \"cluster-samples-operator-6748489656-hlxcf\" (UID: \"6f242b84-732d-4642-b7a9-8e6c874d0bdd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf"
Jul 29 10:29:55.583148 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.583062    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fh4qq"]
Jul 29 10:29:55.586842 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:29:55.586790    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6ba5cdf_2ce5_4286_8a74_6f9f0b89e75d.slice/crio-d43a951da8a4ac169eac830dbddb7e3726d3ec2a098dd6262d3fdeb4104bd02f WatchSource:0}: Error finding container d43a951da8a4ac169eac830dbddb7e3726d3ec2a098dd6262d3fdeb4104bd02f: Status 404 returned error can't find the container with id d43a951da8a4ac169eac830dbddb7e3726d3ec2a098dd6262d3fdeb4104bd02f
Jul 29 10:29:55.603582 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.603544    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5"]
Jul 29 10:29:55.605671 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.605645    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-vm85k"
Jul 29 10:29:55.610845 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:29:55.610794    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22f26925_6582_4316_a1b1_a451fe0ecba8.slice/crio-4c5a71609736ab300c99e7832321c2a3f20e0b5f8d956c0417e588ddc9e1822f WatchSource:0}: Error finding container 4c5a71609736ab300c99e7832321c2a3f20e0b5f8d956c0417e588ddc9e1822f: Status 404 returned error can't find the container with id 4c5a71609736ab300c99e7832321c2a3f20e0b5f8d956c0417e588ddc9e1822f
Jul 29 10:29:55.620790 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.620755    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5" event={"ID":"22f26925-6582-4316-a1b1-a451fe0ecba8","Type":"ContainerStarted","Data":"4c5a71609736ab300c99e7832321c2a3f20e0b5f8d956c0417e588ddc9e1822f"}
Jul 29 10:29:55.622538 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.622508    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fh4qq" event={"ID":"e6ba5cdf-2ce5-4286-8a74-6f9f0b89e75d","Type":"ContainerStarted","Data":"d43a951da8a4ac169eac830dbddb7e3726d3ec2a098dd6262d3fdeb4104bd02f"}
Jul 29 10:29:55.769945 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.769909    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-vm85k"]
Jul 29 10:29:55.774113 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:29:55.774078    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4875bd25_9d39_432d_bffa_2f422607180d.slice/crio-d0b3866f9c9ff3c249e808b2f7a68531836074caefc4a08d61a75a40d7ab4956 WatchSource:0}: Error finding container d0b3866f9c9ff3c249e808b2f7a68531836074caefc4a08d61a75a40d7ab4956: Status 404 returned error can't find the container with id d0b3866f9c9ff3c249e808b2f7a68531836074caefc4a08d61a75a40d7ab4956
Jul 29 10:29:55.820323 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.820280    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:55.820505 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:55.820454    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 10:29:55.820505 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:55.820476    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64ff54b9f5-xd2bg: secret "image-registry-tls" not found
Jul 29 10:29:55.820610 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:55.820550    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls podName:f028e32c-2c95-43b6-907e-79916611f42f nodeName:}" failed. No retries permitted until 2026-07-29 10:29:56.820528023 +0000 UTC m=+68.003884431 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls") pod "image-registry-64ff54b9f5-xd2bg" (UID: "f028e32c-2c95-43b6-907e-79916611f42f") : secret "image-registry-tls" not found
Jul 29 10:29:55.921439 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:55.921313    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hlxcf\" (UID: \"6f242b84-732d-4642-b7a9-8e6c874d0bdd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf"
Jul 29 10:29:55.921617 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:55.921488    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 10:29:55.921617 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:55.921573    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls podName:6f242b84-732d-4642-b7a9-8e6c874d0bdd nodeName:}" failed. No retries permitted until 2026-07-29 10:29:56.92155112 +0000 UTC m=+68.104907516 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls") pod "cluster-samples-operator-6748489656-hlxcf" (UID: "6f242b84-732d-4642-b7a9-8e6c874d0bdd") : secret "samples-operator-tls" not found
Jul 29 10:29:56.630350 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:56.630309    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-vm85k" event={"ID":"4875bd25-9d39-432d-bffa-2f422607180d","Type":"ContainerStarted","Data":"d0b3866f9c9ff3c249e808b2f7a68531836074caefc4a08d61a75a40d7ab4956"}
Jul 29 10:29:56.830473 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:56.830232    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:56.830473 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:56.830420    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 10:29:56.830473 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:56.830436    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64ff54b9f5-xd2bg: secret "image-registry-tls" not found
Jul 29 10:29:56.830767 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:56.830500    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls podName:f028e32c-2c95-43b6-907e-79916611f42f nodeName:}" failed. No retries permitted until 2026-07-29 10:29:58.830480034 +0000 UTC m=+70.013836441 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls") pod "image-registry-64ff54b9f5-xd2bg" (UID: "f028e32c-2c95-43b6-907e-79916611f42f") : secret "image-registry-tls" not found
Jul 29 10:29:56.932209 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:56.931566    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hlxcf\" (UID: \"6f242b84-732d-4642-b7a9-8e6c874d0bdd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf"
Jul 29 10:29:56.932209 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:56.931722    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 10:29:56.932209 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:56.931789    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls podName:6f242b84-732d-4642-b7a9-8e6c874d0bdd nodeName:}" failed. No retries permitted until 2026-07-29 10:29:58.931769123 +0000 UTC m=+70.115125524 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls") pod "cluster-samples-operator-6748489656-hlxcf" (UID: "6f242b84-732d-4642-b7a9-8e6c874d0bdd") : secret "samples-operator-tls" not found
Jul 29 10:29:58.848499 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:58.848457    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:29:58.848966 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:58.848601    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 10:29:58.848966 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:58.848621    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64ff54b9f5-xd2bg: secret "image-registry-tls" not found
Jul 29 10:29:58.848966 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:58.848678    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls podName:f028e32c-2c95-43b6-907e-79916611f42f nodeName:}" failed. No retries permitted until 2026-07-29 10:30:02.848662231 +0000 UTC m=+74.032018625 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls") pod "image-registry-64ff54b9f5-xd2bg" (UID: "f028e32c-2c95-43b6-907e-79916611f42f") : secret "image-registry-tls" not found
Jul 29 10:29:58.949815 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:29:58.949764    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hlxcf\" (UID: \"6f242b84-732d-4642-b7a9-8e6c874d0bdd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf"
Jul 29 10:29:58.949991 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:58.949948    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 10:29:58.950038 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:29:58.950026    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls podName:6f242b84-732d-4642-b7a9-8e6c874d0bdd nodeName:}" failed. No retries permitted until 2026-07-29 10:30:02.950004599 +0000 UTC m=+74.133360995 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls") pod "cluster-samples-operator-6748489656-hlxcf" (UID: "6f242b84-732d-4642-b7a9-8e6c874d0bdd") : secret "samples-operator-tls" not found
Jul 29 10:30:01.644812 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:01.644765    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rvqcw" event={"ID":"a6d54b5d-72a9-412c-8dbc-e4993832122a","Type":"ContainerStarted","Data":"cd1241e09616878e3d3d40479ce44b887064caf31eb873210f36bdf0fc96c270"}
Jul 29 10:30:01.646331 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:01.646301    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5" event={"ID":"22f26925-6582-4316-a1b1-a451fe0ecba8","Type":"ContainerStarted","Data":"ef9963462de86a0d58bb054035c7180203b201a5ec6d604c46450af7bb8b771d"}
Jul 29 10:30:01.647735 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:01.647707    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fp5xd" event={"ID":"33f96a45-1e06-4af4-bcbe-7e494cfcbfe2","Type":"ContainerStarted","Data":"77e503565b13276a74ff35ddc4778275ded0561ddb1fede3f276e7a9447670e7"}
Jul 29 10:30:01.647869 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:01.647762    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:30:01.649278 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:01.649247    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fh4qq" event={"ID":"e6ba5cdf-2ce5-4286-8a74-6f9f0b89e75d","Type":"ContainerStarted","Data":"826d9e1c7dadd89498cbe951a8009ba416e4655db946df70b3dd8e1d7e8ec817"}
Jul 29 10:30:01.650648 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:01.650627    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-vm85k" event={"ID":"4875bd25-9d39-432d-bffa-2f422607180d","Type":"ContainerStarted","Data":"ee25b5bb23a721c135671e72b771bde052eea9a51364de6f95cf09d8d1032837"}
Jul 29 10:30:01.668084 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:01.668031    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rvqcw" podStartSLOduration=66.37365258 podStartE2EDuration="1m12.668015252s" totalImagesPullingTime="6.294362672s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:01.666974743 +0000 UTC m=+72.850331159" watchObservedRunningTime="2026-07-29 10:30:01.668015252 +0000 UTC m=+72.851371644"
Jul 29 10:30:01.690027 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:01.689975    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fp5xd" podStartSLOduration=66.416110408 podStartE2EDuration="1m12.689960075s" totalImagesPullingTime="6.273849667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:01.689264765 +0000 UTC m=+72.872621181" watchObservedRunningTime="2026-07-29 10:30:01.689960075 +0000 UTC m=+72.873316489"
Jul 29 10:30:01.706405 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:01.706352    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5" podStartSLOduration=1.64816423 podStartE2EDuration="6.706338695s" totalImagesPullingTime="5.058174465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:01.70562545 +0000 UTC m=+72.888981892" watchObservedRunningTime="2026-07-29 10:30:01.706338695 +0000 UTC m=+72.889695110"
Jul 29 10:30:01.722243 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:01.722192    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-fh4qq" podStartSLOduration=1.641587326 podStartE2EDuration="6.722176718s" totalImagesPullingTime="5.080589392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:01.721205659 +0000 UTC m=+72.904562093" watchObservedRunningTime="2026-07-29 10:30:01.722176718 +0000 UTC m=+72.905533132"
Jul 29 10:30:01.741556 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:01.741508    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-vm85k" podStartSLOduration=1.8385662759999999 podStartE2EDuration="6.741493362s" totalImagesPullingTime="4.902927086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:01.740533713 +0000 UTC m=+72.923890130" watchObservedRunningTime="2026-07-29 10:30:01.741493362 +0000 UTC m=+72.924849776"
Jul 29 10:30:01.998579 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:01.998544    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-brmw9"]
Jul 29 10:30:02.000905 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.000887    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:02.003855 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.003834    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 10:30:02.004125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.004111    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 10:30:02.004389 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.004358    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-27qr7\""
Jul 29 10:30:02.018927 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.018899    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-brmw9"]
Jul 29 10:30:02.071460 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.071432    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9aad0867-9ee3-4e88-9704-a20cf319dbe8-crio-socket\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:02.071655 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.071470    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:02.071655 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.071614    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9aad0867-9ee3-4e88-9704-a20cf319dbe8-data-volume\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:02.071772 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.071661    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vpkt\" (UniqueName: \"kubernetes.io/projected/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-api-access-8vpkt\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:02.071772 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.071709    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:02.172742 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.172709    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9aad0867-9ee3-4e88-9704-a20cf319dbe8-data-volume\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:02.172742 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.172749    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vpkt\" (UniqueName: \"kubernetes.io/projected/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-api-access-8vpkt\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:02.172982 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.172781    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:02.172982 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:02.172890    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 10:30:02.172982 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:02.172944    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls podName:9aad0867-9ee3-4e88-9704-a20cf319dbe8 nodeName:}" failed. No retries permitted until 2026-07-29 10:30:02.672930332 +0000 UTC m=+73.856286728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-brmw9" (UID: "9aad0867-9ee3-4e88-9704-a20cf319dbe8") : secret "insights-runtime-extractor-tls" not found
Jul 29 10:30:02.172982 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.172956    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9aad0867-9ee3-4e88-9704-a20cf319dbe8-crio-socket\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:02.172982 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.172977    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:02.173225 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.173057    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9aad0867-9ee3-4e88-9704-a20cf319dbe8-crio-socket\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:02.173225 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:02.173070    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 10:30:02.173225 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:02.173110    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm podName:9aad0867-9ee3-4e88-9704-a20cf319dbe8 nodeName:}" failed. No retries permitted until 2026-07-29 10:30:02.67309786 +0000 UTC m=+73.856454269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-brmw9" (UID: "9aad0867-9ee3-4e88-9704-a20cf319dbe8") : configmap "kube-rbac-proxy" not found
Jul 29 10:30:02.173225 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.173109    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9aad0867-9ee3-4e88-9704-a20cf319dbe8-data-volume\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:02.181419 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.181397    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vpkt\" (UniqueName: \"kubernetes.io/projected/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-api-access-8vpkt\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:02.676824 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.676769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:02.677326 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:02.676902    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 10:30:02.677326 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:02.676975    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm podName:9aad0867-9ee3-4e88-9704-a20cf319dbe8 nodeName:}" failed. No retries permitted until 2026-07-29 10:30:03.676954081 +0000 UTC m=+74.860310496 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-brmw9" (UID: "9aad0867-9ee3-4e88-9704-a20cf319dbe8") : configmap "kube-rbac-proxy" not found
Jul 29 10:30:02.677326 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.677146    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:02.677326 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:02.677289    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 10:30:02.677487 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:02.677343    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls podName:9aad0867-9ee3-4e88-9704-a20cf319dbe8 nodeName:}" failed. No retries permitted until 2026-07-29 10:30:03.677329531 +0000 UTC m=+74.860685934 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-brmw9" (UID: "9aad0867-9ee3-4e88-9704-a20cf319dbe8") : secret "insights-runtime-extractor-tls" not found
Jul 29 10:30:02.877993 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.877944    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:30:02.878161 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:02.878110    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 10:30:02.878161 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:02.878131    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64ff54b9f5-xd2bg: secret "image-registry-tls" not found
Jul 29 10:30:02.878247 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:02.878187    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls podName:f028e32c-2c95-43b6-907e-79916611f42f nodeName:}" failed. No retries permitted until 2026-07-29 10:30:10.878172397 +0000 UTC m=+82.061528789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls") pod "image-registry-64ff54b9f5-xd2bg" (UID: "f028e32c-2c95-43b6-907e-79916611f42f") : secret "image-registry-tls" not found
Jul 29 10:30:02.979494 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:02.979461    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hlxcf\" (UID: \"6f242b84-732d-4642-b7a9-8e6c874d0bdd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf"
Jul 29 10:30:02.979635 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:02.979613    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 10:30:02.979687 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:02.979678    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls podName:6f242b84-732d-4642-b7a9-8e6c874d0bdd nodeName:}" failed. No retries permitted until 2026-07-29 10:30:10.979661242 +0000 UTC m=+82.163017636 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls") pod "cluster-samples-operator-6748489656-hlxcf" (UID: "6f242b84-732d-4642-b7a9-8e6c874d0bdd") : secret "samples-operator-tls" not found
Jul 29 10:30:03.685100 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:03.685059    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:03.685512 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:03.685160    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:03.685512 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:03.685186    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 10:30:03.685512 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:03.685249    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm podName:9aad0867-9ee3-4e88-9704-a20cf319dbe8 nodeName:}" failed. No retries permitted until 2026-07-29 10:30:05.685233785 +0000 UTC m=+76.868590178 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-brmw9" (UID: "9aad0867-9ee3-4e88-9704-a20cf319dbe8") : configmap "kube-rbac-proxy" not found
Jul 29 10:30:03.685512 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:03.685278    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 10:30:03.685512 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:03.685316    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls podName:9aad0867-9ee3-4e88-9704-a20cf319dbe8 nodeName:}" failed. No retries permitted until 2026-07-29 10:30:05.685305632 +0000 UTC m=+76.868662025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-brmw9" (UID: "9aad0867-9ee3-4e88-9704-a20cf319dbe8") : secret "insights-runtime-extractor-tls" not found
Jul 29 10:30:04.960261 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:04.960234    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wgrdq_7d9c53a1-f5af-4bfc-81ba-4703f2fc039b/dns-node-resolver/0.log"
Jul 29 10:30:05.702359 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:05.702325    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:05.702569 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:05.702383    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:05.702569 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:05.702492    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 10:30:05.702569 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:05.702500    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 10:30:05.702569 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:05.702543    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm podName:9aad0867-9ee3-4e88-9704-a20cf319dbe8 nodeName:}" failed. No retries permitted until 2026-07-29 10:30:09.70252973 +0000 UTC m=+80.885886122 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-brmw9" (UID: "9aad0867-9ee3-4e88-9704-a20cf319dbe8") : configmap "kube-rbac-proxy" not found
Jul 29 10:30:05.702569 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:05.702555    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls podName:9aad0867-9ee3-4e88-9704-a20cf319dbe8 nodeName:}" failed. No retries permitted until 2026-07-29 10:30:09.702549149 +0000 UTC m=+80.885905541 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-brmw9" (UID: "9aad0867-9ee3-4e88-9704-a20cf319dbe8") : secret "insights-runtime-extractor-tls" not found
Jul 29 10:30:05.762097 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:05.762070    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j5xql_3ffc02ea-68ae-4d60-82a4-01d159cd668c/node-ca/0.log"
Jul 29 10:30:06.360772 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:06.360744    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8dxdp_04cbd6e9-e56f-4e33-bc1a-901eeec3248d/serve-healthcheck-canary/0.log"
Jul 29 10:30:07.361816 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:07.361770    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5_22f26925-6582-4316-a1b1-a451fe0ecba8/kube-storage-version-migrator-operator/0.log"
Jul 29 10:30:09.733364 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:09.733325    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:09.733745 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:09.733411    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:09.733745 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:09.733444    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 10:30:09.733745 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:09.733519    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm podName:9aad0867-9ee3-4e88-9704-a20cf319dbe8 nodeName:}" failed. No retries permitted until 2026-07-29 10:30:17.733502325 +0000 UTC m=+88.916858719 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-brmw9" (UID: "9aad0867-9ee3-4e88-9704-a20cf319dbe8") : configmap "kube-rbac-proxy" not found
Jul 29 10:30:09.733745 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:09.733518    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 10:30:09.733745 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:09.733548    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls podName:9aad0867-9ee3-4e88-9704-a20cf319dbe8 nodeName:}" failed. No retries permitted until 2026-07-29 10:30:17.733541999 +0000 UTC m=+88.916898392 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-brmw9" (UID: "9aad0867-9ee3-4e88-9704-a20cf319dbe8") : secret "insights-runtime-extractor-tls" not found
Jul 29 10:30:10.944366 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:10.944334    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:30:10.946652 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:10.946630    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls\") pod \"image-registry-64ff54b9f5-xd2bg\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") " pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:30:11.045715 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:11.045669    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hlxcf\" (UID: \"6f242b84-732d-4642-b7a9-8e6c874d0bdd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf"
Jul 29 10:30:11.047958 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:11.047936    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f242b84-732d-4642-b7a9-8e6c874d0bdd-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hlxcf\" (UID: \"6f242b84-732d-4642-b7a9-8e6c874d0bdd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf"
Jul 29 10:30:11.101126 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:11.101088    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:30:11.213516 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:11.213442    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf"
Jul 29 10:30:11.232221 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:11.232195    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64ff54b9f5-xd2bg"]
Jul 29 10:30:11.235990 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:30:11.235960    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf028e32c_2c95_43b6_907e_79916611f42f.slice/crio-1e28244ad3c5870f8e0e9adcfdade0660699233d3efdd7162c209ef1157d3d4c WatchSource:0}: Error finding container 1e28244ad3c5870f8e0e9adcfdade0660699233d3efdd7162c209ef1157d3d4c: Status 404 returned error can't find the container with id 1e28244ad3c5870f8e0e9adcfdade0660699233d3efdd7162c209ef1157d3d4c
Jul 29 10:30:11.346547 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:11.346508    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf"]
Jul 29 10:30:11.683013 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:11.682978    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf" event={"ID":"6f242b84-732d-4642-b7a9-8e6c874d0bdd","Type":"ContainerStarted","Data":"9a66ebd385c4e0c3b97136d72b8258fc65d768ebe6bc3ecc6fa733d297074338"}
Jul 29 10:30:11.684568 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:11.684532    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg" event={"ID":"f028e32c-2c95-43b6-907e-79916611f42f","Type":"ContainerStarted","Data":"74a96675f82d81ad3b4d4443c236a7ba53b822d63beb24a2c5b1323adaa55ae0"}
Jul 29 10:30:11.684686 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:11.684573    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg" event={"ID":"f028e32c-2c95-43b6-907e-79916611f42f","Type":"ContainerStarted","Data":"1e28244ad3c5870f8e0e9adcfdade0660699233d3efdd7162c209ef1157d3d4c"}
Jul 29 10:30:11.684686 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:11.684666    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:30:11.708724 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:11.708676    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg" podStartSLOduration=16.708662957 podStartE2EDuration="16.708662957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:11.70740657 +0000 UTC m=+82.890763040" watchObservedRunningTime="2026-07-29 10:30:11.708662957 +0000 UTC m=+82.892019371"
Jul 29 10:30:13.692649 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:13.692607    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf" event={"ID":"6f242b84-732d-4642-b7a9-8e6c874d0bdd","Type":"ContainerStarted","Data":"a2769c3f3d4f4f8a667edfe12506f2a7281f5706c9b6a6ed145e61035bf1b4eb"}
Jul 29 10:30:13.693054 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:13.692672    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf" event={"ID":"6f242b84-732d-4642-b7a9-8e6c874d0bdd","Type":"ContainerStarted","Data":"07aa5beef0807b975aabc6ff992789d2d0f25d1c15bb1ab6801c59f0aabc2e89"}
Jul 29 10:30:13.712546 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:13.712493    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hlxcf" podStartSLOduration=17.246588274 podStartE2EDuration="18.712478245s" totalImagesPullingTime="1.465889971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:13.711888019 +0000 UTC m=+84.895244454" watchObservedRunningTime="2026-07-29 10:30:13.712478245 +0000 UTC m=+84.895834660"
Jul 29 10:30:17.802998 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:17.802963    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:17.803389 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:17.803013    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:17.803515 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:17.803496    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9aad0867-9ee3-4e88-9704-a20cf319dbe8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:17.805281 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:17.805249    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9aad0867-9ee3-4e88-9704-a20cf319dbe8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-brmw9\" (UID: \"9aad0867-9ee3-4e88-9704-a20cf319dbe8\") " pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:17.910263 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:17.910232    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-brmw9"
Jul 29 10:30:18.040710 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:18.040678    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-brmw9"]
Jul 29 10:30:18.043735 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:30:18.043688    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aad0867_9ee3_4e88_9704_a20cf319dbe8.slice/crio-49bbc096ff088f48adf1bf9702a0ee5f074d1cd6b5d20af2be75895b953de935 WatchSource:0}: Error finding container 49bbc096ff088f48adf1bf9702a0ee5f074d1cd6b5d20af2be75895b953de935: Status 404 returned error can't find the container with id 49bbc096ff088f48adf1bf9702a0ee5f074d1cd6b5d20af2be75895b953de935
Jul 29 10:30:18.711854 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:18.711816    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-brmw9" event={"ID":"9aad0867-9ee3-4e88-9704-a20cf319dbe8","Type":"ContainerStarted","Data":"49bbc096ff088f48adf1bf9702a0ee5f074d1cd6b5d20af2be75895b953de935"}
Jul 29 10:30:23.259934 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.259896    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd"]
Jul 29 10:30:23.262925 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.262908    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd"
Jul 29 10:30:23.265601 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.265575    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 29 10:30:23.265912 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.265575    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 29 10:30:23.267011 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.266991    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 29 10:30:23.267131 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.266991    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 29 10:30:23.281611 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.281584    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"]
Jul 29 10:30:23.284511 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.284491    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.287116 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.287091    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 29 10:30:23.287199 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.287142    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 29 10:30:23.287199 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.287178    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 29 10:30:23.287303 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.287094    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 29 10:30:23.289948 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.289890    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd"]
Jul 29 10:30:23.312618 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.312588    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"]
Jul 29 10:30:23.347478 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.347447    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb73cab0-d1a2-4f97-a46d-c5720da76195-tmp\") pod \"klusterlet-addon-workmgr-65b99777f7-d9phd\" (UID: \"bb73cab0-d1a2-4f97-a46d-c5720da76195\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd"
Jul 29 10:30:23.347478 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.347480    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ccc5a65c-b4d0-455b-9857-7e698a4e5201-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.347694 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.347523    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ccc5a65c-b4d0-455b-9857-7e698a4e5201-ca\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.347694 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.347595    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ccc5a65c-b4d0-455b-9857-7e698a4e5201-hub\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.347694 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.347655    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bb73cab0-d1a2-4f97-a46d-c5720da76195-klusterlet-config\") pod \"klusterlet-addon-workmgr-65b99777f7-d9phd\" (UID: \"bb73cab0-d1a2-4f97-a46d-c5720da76195\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd"
Jul 29 10:30:23.347867 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.347696    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ccc5a65c-b4d0-455b-9857-7e698a4e5201-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.347867 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.347718    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssr97\" (UniqueName: \"kubernetes.io/projected/bb73cab0-d1a2-4f97-a46d-c5720da76195-kube-api-access-ssr97\") pod \"klusterlet-addon-workmgr-65b99777f7-d9phd\" (UID: \"bb73cab0-d1a2-4f97-a46d-c5720da76195\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd"
Jul 29 10:30:23.347867 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.347747    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfj7c\" (UniqueName: \"kubernetes.io/projected/ccc5a65c-b4d0-455b-9857-7e698a4e5201-kube-api-access-rfj7c\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.347867 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.347792    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccc5a65c-b4d0-455b-9857-7e698a4e5201-additional-service-ca\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.380395 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.380358    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64ff54b9f5-xd2bg"]
Jul 29 10:30:23.448777 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.448719    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ccc5a65c-b4d0-455b-9857-7e698a4e5201-ca\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.448981 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.448843    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ccc5a65c-b4d0-455b-9857-7e698a4e5201-hub\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.448981 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.448902    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bb73cab0-d1a2-4f97-a46d-c5720da76195-klusterlet-config\") pod \"klusterlet-addon-workmgr-65b99777f7-d9phd\" (UID: \"bb73cab0-d1a2-4f97-a46d-c5720da76195\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd"
Jul 29 10:30:23.448981 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.448936    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ccc5a65c-b4d0-455b-9857-7e698a4e5201-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.448981 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.448965    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssr97\" (UniqueName: \"kubernetes.io/projected/bb73cab0-d1a2-4f97-a46d-c5720da76195-kube-api-access-ssr97\") pod \"klusterlet-addon-workmgr-65b99777f7-d9phd\" (UID: \"bb73cab0-d1a2-4f97-a46d-c5720da76195\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd"
Jul 29 10:30:23.449183 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.448998    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfj7c\" (UniqueName: \"kubernetes.io/projected/ccc5a65c-b4d0-455b-9857-7e698a4e5201-kube-api-access-rfj7c\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.449183 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.449027    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccc5a65c-b4d0-455b-9857-7e698a4e5201-additional-service-ca\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.449183 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.449051    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb73cab0-d1a2-4f97-a46d-c5720da76195-tmp\") pod \"klusterlet-addon-workmgr-65b99777f7-d9phd\" (UID: \"bb73cab0-d1a2-4f97-a46d-c5720da76195\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd"
Jul 29 10:30:23.449183 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.449074    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ccc5a65c-b4d0-455b-9857-7e698a4e5201-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.450900 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.449997    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb73cab0-d1a2-4f97-a46d-c5720da76195-tmp\") pod \"klusterlet-addon-workmgr-65b99777f7-d9phd\" (UID: \"bb73cab0-d1a2-4f97-a46d-c5720da76195\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd"
Jul 29 10:30:23.450900 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.450851    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccc5a65c-b4d0-455b-9857-7e698a4e5201-additional-service-ca\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.451778 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.451754    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ccc5a65c-b4d0-455b-9857-7e698a4e5201-ca\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.451897 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.451882    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ccc5a65c-b4d0-455b-9857-7e698a4e5201-hub\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.451946 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.451930    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ccc5a65c-b4d0-455b-9857-7e698a4e5201-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.451987 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.451943    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bb73cab0-d1a2-4f97-a46d-c5720da76195-klusterlet-config\") pod \"klusterlet-addon-workmgr-65b99777f7-d9phd\" (UID: \"bb73cab0-d1a2-4f97-a46d-c5720da76195\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd"
Jul 29 10:30:23.452032 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.452016    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ccc5a65c-b4d0-455b-9857-7e698a4e5201-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.461924 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.461895    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssr97\" (UniqueName: \"kubernetes.io/projected/bb73cab0-d1a2-4f97-a46d-c5720da76195-kube-api-access-ssr97\") pod \"klusterlet-addon-workmgr-65b99777f7-d9phd\" (UID: \"bb73cab0-d1a2-4f97-a46d-c5720da76195\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd"
Jul 29 10:30:23.464239 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.464216    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfj7c\" (UniqueName: \"kubernetes.io/projected/ccc5a65c-b4d0-455b-9857-7e698a4e5201-kube-api-access-rfj7c\") pod \"cluster-proxy-proxy-agent-58dcc4bc5f-8k287\" (UID: \"ccc5a65c-b4d0-455b-9857-7e698a4e5201\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.572917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.572585    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd"
Jul 29 10:30:23.604990 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.604954    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"
Jul 29 10:30:23.733355 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.733252    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-brmw9" event={"ID":"9aad0867-9ee3-4e88-9704-a20cf319dbe8","Type":"ContainerStarted","Data":"a0ce6cd2d71702bfce23d6a81afd08d34fb7904ac6104d3b67097a118d5a3385"}
Jul 29 10:30:23.741000 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.740969    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd"]
Jul 29 10:30:23.744932 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:30:23.744906    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb73cab0_d1a2_4f97_a46d_c5720da76195.slice/crio-042ad6babb30d6c6a2bf6c9deb2c3791573ac4c1b0584e02f53e30f384000f68 WatchSource:0}: Error finding container 042ad6babb30d6c6a2bf6c9deb2c3791573ac4c1b0584e02f53e30f384000f68: Status 404 returned error can't find the container with id 042ad6babb30d6c6a2bf6c9deb2c3791573ac4c1b0584e02f53e30f384000f68
Jul 29 10:30:23.815126 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:23.815096    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287"]
Jul 29 10:30:23.818250 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:30:23.818215    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podccc5a65c_b4d0_455b_9857_7e698a4e5201.slice/crio-6fb31a910267150b50101f07ee95b124ed1d67c96abd77e3757b09d9b55018a7 WatchSource:0}: Error finding container 6fb31a910267150b50101f07ee95b124ed1d67c96abd77e3757b09d9b55018a7: Status 404 returned error can't find the container with id 6fb31a910267150b50101f07ee95b124ed1d67c96abd77e3757b09d9b55018a7
Jul 29 10:30:24.741278 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:24.741235    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287" event={"ID":"ccc5a65c-b4d0-455b-9857-7e698a4e5201","Type":"ContainerStarted","Data":"6fb31a910267150b50101f07ee95b124ed1d67c96abd77e3757b09d9b55018a7"}
Jul 29 10:30:24.743429 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:24.743394    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd" event={"ID":"bb73cab0-d1a2-4f97-a46d-c5720da76195","Type":"ContainerStarted","Data":"042ad6babb30d6c6a2bf6c9deb2c3791573ac4c1b0584e02f53e30f384000f68"}
Jul 29 10:30:27.083741 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:27.083702    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:30:27.086386 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:27.086360    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/73102ac7-c5cf-43b4-a289-4976d3e902e9-metrics-tls\") pod \"dns-default-hhbsk\" (UID: \"73102ac7-c5cf-43b4-a289-4976d3e902e9\") " pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:30:27.100960 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:27.100937    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hm7tz\""
Jul 29 10:30:27.109088 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:27.109057    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:30:30.622823 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:30.622738    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hhbsk"]
Jul 29 10:30:30.625706 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:30:30.625464    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73102ac7_c5cf_43b4_a289_4976d3e902e9.slice/crio-a42c35e7fa8264e452ecec63f4353f058cb108e185629ecabfc443cd3f334ea4 WatchSource:0}: Error finding container a42c35e7fa8264e452ecec63f4353f058cb108e185629ecabfc443cd3f334ea4: Status 404 returned error can't find the container with id a42c35e7fa8264e452ecec63f4353f058cb108e185629ecabfc443cd3f334ea4
Jul 29 10:30:30.766607 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:30.766572    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd" event={"ID":"bb73cab0-d1a2-4f97-a46d-c5720da76195","Type":"ContainerStarted","Data":"7800adf9c72e837f9e830bb479e70a716dbd4ec38c36f258618a74a44669c267"}
Jul 29 10:30:30.766765 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:30.766747    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd"
Jul 29 10:30:30.767860 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:30.767827    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hhbsk" event={"ID":"73102ac7-c5cf-43b4-a289-4976d3e902e9","Type":"ContainerStarted","Data":"a42c35e7fa8264e452ecec63f4353f058cb108e185629ecabfc443cd3f334ea4"}
Jul 29 10:30:30.768445 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:30.768427    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd"
Jul 29 10:30:30.769898 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:30.769874    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287" event={"ID":"ccc5a65c-b4d0-455b-9857-7e698a4e5201","Type":"ContainerStarted","Data":"a7c131a3837201b38bc57ba3550af7928edbf1bc12fa3e5c37cb2e1e7335af95"}
Jul 29 10:30:30.769973 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:30.769911    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287" event={"ID":"ccc5a65c-b4d0-455b-9857-7e698a4e5201","Type":"ContainerStarted","Data":"38ba16bb99935fc8558918d6e77a2329cfeb180bb50a9c8ee33ce789ea2332a5"}
Jul 29 10:30:30.771893 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:30.771863    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-brmw9" event={"ID":"9aad0867-9ee3-4e88-9704-a20cf319dbe8","Type":"ContainerStarted","Data":"de1ffea8889d73fe83d348feeea5224d3ac488880a74f79fb1c506b3ae7306ad"}
Jul 29 10:30:30.783770 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:30.783729    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65b99777f7-d9phd" podStartSLOduration=1.031276749 podStartE2EDuration="7.783714316s" totalImagesPullingTime="6.752437567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:30:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:30.78305288 +0000 UTC m=+101.966409320" watchObservedRunningTime="2026-07-29 10:30:30.783714316 +0000 UTC m=+101.967070730"
Jul 29 10:30:31.778169 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:31.778128    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287" event={"ID":"ccc5a65c-b4d0-455b-9857-7e698a4e5201","Type":"ContainerStarted","Data":"303dfb5a01d65e8bad80bc95f0baabd44cd08f325e5c90aa574067e6cd28e6e3"}
Jul 29 10:30:31.803063 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:31.803006    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58dcc4bc5f-8k287" podStartSLOduration=2.140335909 podStartE2EDuration="8.802961888s" totalImagesPullingTime="6.662625979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:30:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:31.801437019 +0000 UTC m=+102.984793445" watchObservedRunningTime="2026-07-29 10:30:31.802961888 +0000 UTC m=+102.986318304"
Jul 29 10:30:32.656167 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:32.656136    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fp5xd"
Jul 29 10:30:33.386234 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:33.386206    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:30:34.791263 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:34.791226    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hhbsk" event={"ID":"73102ac7-c5cf-43b4-a289-4976d3e902e9","Type":"ContainerStarted","Data":"55b68f782fa66cf4d00c2c78290f29d8c1f3a8dc783d2b2c48ed66057497e7d2"}
Jul 29 10:30:35.796779 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:35.796740    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-brmw9" event={"ID":"9aad0867-9ee3-4e88-9704-a20cf319dbe8","Type":"ContainerStarted","Data":"c74662608651554109db06c0f472284e41ddeb8125d996af843566fe69de353e"}
Jul 29 10:30:35.798276 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:35.798252    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hhbsk" event={"ID":"73102ac7-c5cf-43b4-a289-4976d3e902e9","Type":"ContainerStarted","Data":"1d95812409d9b7afe22873e37dd38be0beb2a965b78125699f6a9b509f5cca00"}
Jul 29 10:30:35.798387 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:35.798372    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:30:35.816729 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:35.816683    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-brmw9" podStartSLOduration=18.087485511 podStartE2EDuration="34.81666779s" totalImagesPullingTime="16.729182279s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:30:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:35.814927664 +0000 UTC m=+106.998284090" watchObservedRunningTime="2026-07-29 10:30:35.81666779 +0000 UTC m=+107.000024222"
Jul 29 10:30:35.832442 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:35.832390    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hhbsk" podStartSLOduration=68.790262129 podStartE2EDuration="1m12.832373727s" totalImagesPullingTime="4.042111598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:35.831509245 +0000 UTC m=+107.014865660" watchObservedRunningTime="2026-07-29 10:30:35.832373727 +0000 UTC m=+107.015730142"
Jul 29 10:30:36.435349 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.435310    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6ps2g"]
Jul 29 10:30:36.437672 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.437652    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.440495 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.440473    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 10:30:36.440748 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.440729    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 10:30:36.441825 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.441791    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 10:30:36.441825 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.441817    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-22g7n\""
Jul 29 10:30:36.441981 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.441896    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 10:30:36.441981 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.441896    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 10:30:36.441981 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.441921    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 10:30:36.561780 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.561739    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.561780 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.561776    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-accelerators-collector-config\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.562018 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.561880    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-tls\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.562018 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.561911    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-wtmp\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.562018 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.561935    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4decb700-a44b-48d9-9e6d-726e45aac336-sys\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.562018 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.561952    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74gn8\" (UniqueName: \"kubernetes.io/projected/4decb700-a44b-48d9-9e6d-726e45aac336-kube-api-access-74gn8\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.562018 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.562003    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-textfile\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.562185 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.562045    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4decb700-a44b-48d9-9e6d-726e45aac336-root\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.562185 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.562064    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4decb700-a44b-48d9-9e6d-726e45aac336-metrics-client-ca\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.663347 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.663312    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.663347 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.663348    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-accelerators-collector-config\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.663595 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.663393    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-tls\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.663595 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.663411    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-wtmp\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.663595 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.663427    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4decb700-a44b-48d9-9e6d-726e45aac336-sys\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.663595 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.663442    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74gn8\" (UniqueName: \"kubernetes.io/projected/4decb700-a44b-48d9-9e6d-726e45aac336-kube-api-access-74gn8\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.663595 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.663460    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-textfile\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.663595 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.663513    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4decb700-a44b-48d9-9e6d-726e45aac336-root\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.663595 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.663547    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4decb700-a44b-48d9-9e6d-726e45aac336-sys\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.663595 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.663556    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4decb700-a44b-48d9-9e6d-726e45aac336-metrics-client-ca\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.663595 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.663562    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4decb700-a44b-48d9-9e6d-726e45aac336-root\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.663595 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:36.663575    2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 29 10:30:36.664161 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:36.663649    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-tls podName:4decb700-a44b-48d9-9e6d-726e45aac336 nodeName:}" failed. No retries permitted until 2026-07-29 10:30:37.163628927 +0000 UTC m=+108.346985342 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-tls") pod "node-exporter-6ps2g" (UID: "4decb700-a44b-48d9-9e6d-726e45aac336") : secret "node-exporter-tls" not found
Jul 29 10:30:36.664161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.663685    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-wtmp\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.664161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.663981    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-textfile\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.664161 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.664106    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4decb700-a44b-48d9-9e6d-726e45aac336-metrics-client-ca\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.664320 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.664187    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-accelerators-collector-config\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.665862 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.665841    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:36.672135 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:36.672115    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74gn8\" (UniqueName: \"kubernetes.io/projected/4decb700-a44b-48d9-9e6d-726e45aac336-kube-api-access-74gn8\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:37.168784 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:37.168741    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-tls\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:37.169175 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:37.168904    2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 29 10:30:37.169175 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:37.168970    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-tls podName:4decb700-a44b-48d9-9e6d-726e45aac336 nodeName:}" failed. No retries permitted until 2026-07-29 10:30:38.168954535 +0000 UTC m=+109.352310928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-tls") pod "node-exporter-6ps2g" (UID: "4decb700-a44b-48d9-9e6d-726e45aac336") : secret "node-exporter-tls" not found
Jul 29 10:30:38.176642 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:38.176591    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-tls\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:38.178955 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:38.178930    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4decb700-a44b-48d9-9e6d-726e45aac336-node-exporter-tls\") pod \"node-exporter-6ps2g\" (UID: \"4decb700-a44b-48d9-9e6d-726e45aac336\") " pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:38.246712 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:38.246674    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6ps2g"
Jul 29 10:30:38.256119 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:30:38.256094    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4decb700_a44b_48d9_9e6d_726e45aac336.slice/crio-7c021c09361a8a6b4a1d693d4669815058cad6f47c01112ee4152a3e5e8f265d WatchSource:0}: Error finding container 7c021c09361a8a6b4a1d693d4669815058cad6f47c01112ee4152a3e5e8f265d: Status 404 returned error can't find the container with id 7c021c09361a8a6b4a1d693d4669815058cad6f47c01112ee4152a3e5e8f265d
Jul 29 10:30:38.811073 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:38.811029    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6ps2g" event={"ID":"4decb700-a44b-48d9-9e6d-726e45aac336","Type":"ContainerStarted","Data":"7c021c09361a8a6b4a1d693d4669815058cad6f47c01112ee4152a3e5e8f265d"}
Jul 29 10:30:39.816298 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:39.816265    2564 generic.go:356] "Generic (PLEG): container finished" podID="4decb700-a44b-48d9-9e6d-726e45aac336" containerID="18c0f0e8e51d828fd5786ae1e32bd4da719117a3f9883e58a3f47982fc75a484" exitCode=0
Jul 29 10:30:39.816670 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:39.816342    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6ps2g" event={"ID":"4decb700-a44b-48d9-9e6d-726e45aac336","Type":"ContainerDied","Data":"18c0f0e8e51d828fd5786ae1e32bd4da719117a3f9883e58a3f47982fc75a484"}
Jul 29 10:30:40.821658 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:40.821621    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6ps2g" event={"ID":"4decb700-a44b-48d9-9e6d-726e45aac336","Type":"ContainerStarted","Data":"1d9785be2c9b7f932abb96370ca64d04878a71653d2f2f863e5a7a4a9ef3598b"}
Jul 29 10:30:40.822056 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:40.821685    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6ps2g" event={"ID":"4decb700-a44b-48d9-9e6d-726e45aac336","Type":"ContainerStarted","Data":"a6de80817d31abc7370066a2515196b6b2877338f3f4492f321e5ed5c80038a8"}
Jul 29 10:30:40.842325 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:40.842276    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6ps2g" podStartSLOduration=3.928020295 podStartE2EDuration="4.842256811s" totalImagesPullingTime="849.795085ms" totalInitContainerRuntime="64.441431ms" isStatefulPod=true podCreationTimestamp="2026-07-29 10:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:40.840789823 +0000 UTC m=+112.024146237" watchObservedRunningTime="2026-07-29 10:30:40.842256811 +0000 UTC m=+112.025613225"
Jul 29 10:30:45.806963 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:45.806925    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hhbsk"
Jul 29 10:30:48.402685 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.402617    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg" podUID="f028e32c-2c95-43b6-907e-79916611f42f" containerName="registry" containerID="cri-o://74a96675f82d81ad3b4d4443c236a7ba53b822d63beb24a2c5b1323adaa55ae0" gracePeriod=30
Jul 29 10:30:48.634016 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.633990    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:30:48.654419 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.654347    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f028e32c-2c95-43b6-907e-79916611f42f-trusted-ca\") pod \"f028e32c-2c95-43b6-907e-79916611f42f\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") "
Jul 29 10:30:48.654419 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.654409    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f028e32c-2c95-43b6-907e-79916611f42f-ca-trust-extracted\") pod \"f028e32c-2c95-43b6-907e-79916611f42f\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") "
Jul 29 10:30:48.654617 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.654448    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f028e32c-2c95-43b6-907e-79916611f42f-installation-pull-secrets\") pod \"f028e32c-2c95-43b6-907e-79916611f42f\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") "
Jul 29 10:30:48.654617 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.654560    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhcn7\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-kube-api-access-vhcn7\") pod \"f028e32c-2c95-43b6-907e-79916611f42f\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") "
Jul 29 10:30:48.657125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.654615    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-bound-sa-token\") pod \"f028e32c-2c95-43b6-907e-79916611f42f\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") "
Jul 29 10:30:48.657125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.654645    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls\") pod \"f028e32c-2c95-43b6-907e-79916611f42f\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") "
Jul 29 10:30:48.657125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.654679    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f028e32c-2c95-43b6-907e-79916611f42f-registry-certificates\") pod \"f028e32c-2c95-43b6-907e-79916611f42f\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") "
Jul 29 10:30:48.657125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.654727    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f028e32c-2c95-43b6-907e-79916611f42f-image-registry-private-configuration\") pod \"f028e32c-2c95-43b6-907e-79916611f42f\" (UID: \"f028e32c-2c95-43b6-907e-79916611f42f\") "
Jul 29 10:30:48.657125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.654870    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f028e32c-2c95-43b6-907e-79916611f42f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f028e32c-2c95-43b6-907e-79916611f42f" (UID: "f028e32c-2c95-43b6-907e-79916611f42f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:30:48.657125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.655108    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f028e32c-2c95-43b6-907e-79916611f42f-trusted-ca\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:30:48.657125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.655391    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f028e32c-2c95-43b6-907e-79916611f42f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f028e32c-2c95-43b6-907e-79916611f42f" (UID: "f028e32c-2c95-43b6-907e-79916611f42f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:30:48.658072 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.658026    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f028e32c-2c95-43b6-907e-79916611f42f" (UID: "f028e32c-2c95-43b6-907e-79916611f42f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:30:48.658773 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.658650    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f028e32c-2c95-43b6-907e-79916611f42f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f028e32c-2c95-43b6-907e-79916611f42f" (UID: "f028e32c-2c95-43b6-907e-79916611f42f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:30:48.658773 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.658693    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f028e32c-2c95-43b6-907e-79916611f42f" (UID: "f028e32c-2c95-43b6-907e-79916611f42f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:30:48.659197 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.659164    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f028e32c-2c95-43b6-907e-79916611f42f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f028e32c-2c95-43b6-907e-79916611f42f" (UID: "f028e32c-2c95-43b6-907e-79916611f42f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:30:48.659740 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.659711    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-kube-api-access-vhcn7" (OuterVolumeSpecName: "kube-api-access-vhcn7") pod "f028e32c-2c95-43b6-907e-79916611f42f" (UID: "f028e32c-2c95-43b6-907e-79916611f42f"). InnerVolumeSpecName "kube-api-access-vhcn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:30:48.666278 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.666249    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f028e32c-2c95-43b6-907e-79916611f42f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f028e32c-2c95-43b6-907e-79916611f42f" (UID: "f028e32c-2c95-43b6-907e-79916611f42f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 10:30:48.755966 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.755937    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f028e32c-2c95-43b6-907e-79916611f42f-installation-pull-secrets\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:30:48.755966 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.755963    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhcn7\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-kube-api-access-vhcn7\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:30:48.755966 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.755974    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-bound-sa-token\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:30:48.756212 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.755983    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f028e32c-2c95-43b6-907e-79916611f42f-registry-tls\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:30:48.756212 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.755992    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f028e32c-2c95-43b6-907e-79916611f42f-registry-certificates\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:30:48.756212 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.756001    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f028e32c-2c95-43b6-907e-79916611f42f-image-registry-private-configuration\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:30:48.756212 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.756011    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f028e32c-2c95-43b6-907e-79916611f42f-ca-trust-extracted\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:30:48.849513 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.849477    2564 generic.go:356] "Generic (PLEG): container finished" podID="f028e32c-2c95-43b6-907e-79916611f42f" containerID="74a96675f82d81ad3b4d4443c236a7ba53b822d63beb24a2c5b1323adaa55ae0" exitCode=0
Jul 29 10:30:48.849673 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.849521    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg"
Jul 29 10:30:48.849673 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.849570    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg" event={"ID":"f028e32c-2c95-43b6-907e-79916611f42f","Type":"ContainerDied","Data":"74a96675f82d81ad3b4d4443c236a7ba53b822d63beb24a2c5b1323adaa55ae0"}
Jul 29 10:30:48.849673 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.849643    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64ff54b9f5-xd2bg" event={"ID":"f028e32c-2c95-43b6-907e-79916611f42f","Type":"ContainerDied","Data":"1e28244ad3c5870f8e0e9adcfdade0660699233d3efdd7162c209ef1157d3d4c"}
Jul 29 10:30:48.849673 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.849664    2564 scope.go:117] "RemoveContainer" containerID="74a96675f82d81ad3b4d4443c236a7ba53b822d63beb24a2c5b1323adaa55ae0"
Jul 29 10:30:48.858637 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.858619    2564 scope.go:117] "RemoveContainer" containerID="74a96675f82d81ad3b4d4443c236a7ba53b822d63beb24a2c5b1323adaa55ae0"
Jul 29 10:30:48.858912 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:30:48.858892    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74a96675f82d81ad3b4d4443c236a7ba53b822d63beb24a2c5b1323adaa55ae0\": container with ID starting with 74a96675f82d81ad3b4d4443c236a7ba53b822d63beb24a2c5b1323adaa55ae0 not found: ID does not exist" containerID="74a96675f82d81ad3b4d4443c236a7ba53b822d63beb24a2c5b1323adaa55ae0"
Jul 29 10:30:48.858979 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.858919    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74a96675f82d81ad3b4d4443c236a7ba53b822d63beb24a2c5b1323adaa55ae0"} err="failed to get container status \"74a96675f82d81ad3b4d4443c236a7ba53b822d63beb24a2c5b1323adaa55ae0\": rpc error: code = NotFound desc = could not find container \"74a96675f82d81ad3b4d4443c236a7ba53b822d63beb24a2c5b1323adaa55ae0\": container with ID starting with 74a96675f82d81ad3b4d4443c236a7ba53b822d63beb24a2c5b1323adaa55ae0 not found: ID does not exist"
Jul 29 10:30:48.870547 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.870525    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64ff54b9f5-xd2bg"]
Jul 29 10:30:48.873391 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:48.873373    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-64ff54b9f5-xd2bg"]
Jul 29 10:30:49.342594 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:49.342559    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f028e32c-2c95-43b6-907e-79916611f42f" path="/var/lib/kubelet/pods/f028e32c-2c95-43b6-907e-79916611f42f/volumes"
Jul 29 10:30:58.124970 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:58.124932    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs\") pod \"network-metrics-daemon-x5nh2\" (UID: \"d598bd41-0d98-42fb-b082-8582a3b3b5ef\") " pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:30:58.127213 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:58.127179    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d598bd41-0d98-42fb-b082-8582a3b3b5ef-metrics-certs\") pod \"network-metrics-daemon-x5nh2\" (UID: \"d598bd41-0d98-42fb-b082-8582a3b3b5ef\") " pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:30:58.152692 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:58.152665    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n9sqp\""
Jul 29 10:30:58.159862 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:58.159837    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x5nh2"
Jul 29 10:30:58.299055 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:58.299019    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x5nh2"]
Jul 29 10:30:58.301888 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:30:58.301863    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd598bd41_0d98_42fb_b082_8582a3b3b5ef.slice/crio-39c3383aca1045b2f582bad9b7c3eea62ee5e556753165c00aadef49932ccee7 WatchSource:0}: Error finding container 39c3383aca1045b2f582bad9b7c3eea62ee5e556753165c00aadef49932ccee7: Status 404 returned error can't find the container with id 39c3383aca1045b2f582bad9b7c3eea62ee5e556753165c00aadef49932ccee7
Jul 29 10:30:58.884050 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:58.884002    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x5nh2" event={"ID":"d598bd41-0d98-42fb-b082-8582a3b3b5ef","Type":"ContainerStarted","Data":"39c3383aca1045b2f582bad9b7c3eea62ee5e556753165c00aadef49932ccee7"}
Jul 29 10:30:59.889180 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:59.889145    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x5nh2" event={"ID":"d598bd41-0d98-42fb-b082-8582a3b3b5ef","Type":"ContainerStarted","Data":"1014f1a3f9038670c772d97a86e0119e2b5d4228b87232d15fb68fab29bc38f0"}
Jul 29 10:30:59.889567 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:59.889206    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x5nh2" event={"ID":"d598bd41-0d98-42fb-b082-8582a3b3b5ef","Type":"ContainerStarted","Data":"b47c06d91938e5facc4b36c72d5c0f14e9f84c2020749e1c62f53447cdda9b23"}
Jul 29 10:30:59.908326 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:30:59.908278    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-x5nh2" podStartSLOduration=130.037223283 podStartE2EDuration="2m10.908262872s" totalImagesPullingTime="871.039589ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:59.907344305 +0000 UTC m=+131.090700719" watchObservedRunningTime="2026-07-29 10:30:59.908262872 +0000 UTC m=+131.091619317"
Jul 29 10:31:21.970326 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:31:21.970289    2564 generic.go:356] "Generic (PLEG): container finished" podID="22f26925-6582-4316-a1b1-a451fe0ecba8" containerID="ef9963462de86a0d58bb054035c7180203b201a5ec6d604c46450af7bb8b771d" exitCode=0
Jul 29 10:31:21.970824 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:31:21.970365    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5" event={"ID":"22f26925-6582-4316-a1b1-a451fe0ecba8","Type":"ContainerDied","Data":"ef9963462de86a0d58bb054035c7180203b201a5ec6d604c46450af7bb8b771d"}
Jul 29 10:31:21.970824 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:31:21.970745    2564 scope.go:117] "RemoveContainer" containerID="ef9963462de86a0d58bb054035c7180203b201a5ec6d604c46450af7bb8b771d"
Jul 29 10:31:21.972197 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:31:21.972109    2564 generic.go:356] "Generic (PLEG): container finished" podID="4875bd25-9d39-432d-bffa-2f422607180d" containerID="ee25b5bb23a721c135671e72b771bde052eea9a51364de6f95cf09d8d1032837" exitCode=0
Jul 29 10:31:21.972197 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:31:21.972160    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-vm85k" event={"ID":"4875bd25-9d39-432d-bffa-2f422607180d","Type":"ContainerDied","Data":"ee25b5bb23a721c135671e72b771bde052eea9a51364de6f95cf09d8d1032837"}
Jul 29 10:31:21.972509 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:31:21.972492    2564 scope.go:117] "RemoveContainer" containerID="ee25b5bb23a721c135671e72b771bde052eea9a51364de6f95cf09d8d1032837"
Jul 29 10:31:22.977988 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:31:22.977952    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5" event={"ID":"22f26925-6582-4316-a1b1-a451fe0ecba8","Type":"ContainerStarted","Data":"494169260f688560b808ce8fe1c0661d3fcaa89ace613650f354abdc9a28de68"}
Jul 29 10:31:22.979609 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:31:22.979586    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-vm85k" event={"ID":"4875bd25-9d39-432d-bffa-2f422607180d","Type":"ContainerStarted","Data":"42722852f0e99bffc9814a09e307a807e9ff6de3d29aa6b792be9272f820c522"}
Jul 29 10:33:49.224019 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:33:49.223990    2564 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 10:35:02.427721 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.427638    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p"]
Jul 29 10:35:02.428190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.427911    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f028e32c-2c95-43b6-907e-79916611f42f" containerName="registry"
Jul 29 10:35:02.428190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.427923    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f028e32c-2c95-43b6-907e-79916611f42f" containerName="registry"
Jul 29 10:35:02.428190 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.427973    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f028e32c-2c95-43b6-907e-79916611f42f" containerName="registry"
Jul 29 10:35:02.430660 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.430644    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p"
Jul 29 10:35:02.433195 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.433178    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 29 10:35:02.433641 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.433626    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 29 10:35:02.433696 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.433626    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 29 10:35:02.433765 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.433676    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-8kmmd\""
Jul 29 10:35:02.444509 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.444484    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p"]
Jul 29 10:35:02.464718 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.464688    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/4045e0e2-29d3-4d91-9a71-8433c5373115-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p\" (UID: \"4045e0e2-29d3-4d91-9a71-8433c5373115\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p"
Jul 29 10:35:02.464890 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.464726    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncj7k\" (UniqueName: \"kubernetes.io/projected/4045e0e2-29d3-4d91-9a71-8433c5373115-kube-api-access-ncj7k\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p\" (UID: \"4045e0e2-29d3-4d91-9a71-8433c5373115\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p"
Jul 29 10:35:02.565284 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.565249    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/4045e0e2-29d3-4d91-9a71-8433c5373115-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p\" (UID: \"4045e0e2-29d3-4d91-9a71-8433c5373115\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p"
Jul 29 10:35:02.565454 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.565291    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncj7k\" (UniqueName: \"kubernetes.io/projected/4045e0e2-29d3-4d91-9a71-8433c5373115-kube-api-access-ncj7k\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p\" (UID: \"4045e0e2-29d3-4d91-9a71-8433c5373115\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p"
Jul 29 10:35:02.567637 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.567615    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/4045e0e2-29d3-4d91-9a71-8433c5373115-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p\" (UID: \"4045e0e2-29d3-4d91-9a71-8433c5373115\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p"
Jul 29 10:35:02.573716 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.573692    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncj7k\" (UniqueName: \"kubernetes.io/projected/4045e0e2-29d3-4d91-9a71-8433c5373115-kube-api-access-ncj7k\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p\" (UID: \"4045e0e2-29d3-4d91-9a71-8433c5373115\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p"
Jul 29 10:35:02.740619 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.740583    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p"
Jul 29 10:35:02.878340 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.878305    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p"]
Jul 29 10:35:02.882057 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:35:02.882030    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4045e0e2_29d3_4d91_9a71_8433c5373115.slice/crio-9be186d85cfd78610553dd3a6de4309adb221ce65662926113a3970590cd9fa9 WatchSource:0}: Error finding container 9be186d85cfd78610553dd3a6de4309adb221ce65662926113a3970590cd9fa9: Status 404 returned error can't find the container with id 9be186d85cfd78610553dd3a6de4309adb221ce65662926113a3970590cd9fa9
Jul 29 10:35:02.884049 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:02.884028    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 10:35:03.707458 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:03.707422    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p" event={"ID":"4045e0e2-29d3-4d91-9a71-8433c5373115","Type":"ContainerStarted","Data":"9be186d85cfd78610553dd3a6de4309adb221ce65662926113a3970590cd9fa9"}
Jul 29 10:35:23.700057 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.700023    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-77f648d694-zgfb5"]
Jul 29 10:35:23.703101 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.703079    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-zgfb5"
Jul 29 10:35:23.705585 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.705562    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 29 10:35:23.705693 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.705562    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Jul 29 10:35:23.705693 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.705660    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-x5b5q\""
Jul 29 10:35:23.713245 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.713224    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-zgfb5"]
Jul 29 10:35:23.777680 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.777646    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p" event={"ID":"4045e0e2-29d3-4d91-9a71-8433c5373115","Type":"ContainerStarted","Data":"713b0cfa8677929ecf91ba986d7b9cef7dd5564c9551d154670d7d02031042d8"}
Jul 29 10:35:23.777873 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.777724    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p"
Jul 29 10:35:23.796883 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.796836    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p" podStartSLOduration=1.651607701 podStartE2EDuration="21.796819759s" totalImagesPullingTime="20.145212058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:35:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:35:23.795823705 +0000 UTC m=+394.979180115" watchObservedRunningTime="2026-07-29 10:35:23.796819759 +0000 UTC m=+394.980176164"
Jul 29 10:35:23.823188 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.823150    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/6b99c00e-deb7-4632-8caf-06e2ccfa05b2-cabundle0\") pod \"keda-operator-77f648d694-zgfb5\" (UID: \"6b99c00e-deb7-4632-8caf-06e2ccfa05b2\") " pod="openshift-keda/keda-operator-77f648d694-zgfb5"
Jul 29 10:35:23.823368 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.823201    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6b99c00e-deb7-4632-8caf-06e2ccfa05b2-certificates\") pod \"keda-operator-77f648d694-zgfb5\" (UID: \"6b99c00e-deb7-4632-8caf-06e2ccfa05b2\") " pod="openshift-keda/keda-operator-77f648d694-zgfb5"
Jul 29 10:35:23.823368 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.823262    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7pjx\" (UniqueName: \"kubernetes.io/projected/6b99c00e-deb7-4632-8caf-06e2ccfa05b2-kube-api-access-g7pjx\") pod \"keda-operator-77f648d694-zgfb5\" (UID: \"6b99c00e-deb7-4632-8caf-06e2ccfa05b2\") " pod="openshift-keda/keda-operator-77f648d694-zgfb5"
Jul 29 10:35:23.924557 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.924514    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/6b99c00e-deb7-4632-8caf-06e2ccfa05b2-cabundle0\") pod \"keda-operator-77f648d694-zgfb5\" (UID: \"6b99c00e-deb7-4632-8caf-06e2ccfa05b2\") " pod="openshift-keda/keda-operator-77f648d694-zgfb5"
Jul 29 10:35:23.924742 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.924582    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6b99c00e-deb7-4632-8caf-06e2ccfa05b2-certificates\") pod \"keda-operator-77f648d694-zgfb5\" (UID: \"6b99c00e-deb7-4632-8caf-06e2ccfa05b2\") " pod="openshift-keda/keda-operator-77f648d694-zgfb5"
Jul 29 10:35:23.924742 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.924613    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7pjx\" (UniqueName: \"kubernetes.io/projected/6b99c00e-deb7-4632-8caf-06e2ccfa05b2-kube-api-access-g7pjx\") pod \"keda-operator-77f648d694-zgfb5\" (UID: \"6b99c00e-deb7-4632-8caf-06e2ccfa05b2\") " pod="openshift-keda/keda-operator-77f648d694-zgfb5"
Jul 29 10:35:23.924880 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:35:23.924735    2564 secret.go:281] references non-existent secret key: ca.crt
Jul 29 10:35:23.924880 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:35:23.924757    2564 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 29 10:35:23.924880 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:35:23.924768    2564 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-zgfb5: references non-existent secret key: ca.crt
Jul 29 10:35:23.924880 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:35:23.924835    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b99c00e-deb7-4632-8caf-06e2ccfa05b2-certificates podName:6b99c00e-deb7-4632-8caf-06e2ccfa05b2 nodeName:}" failed. No retries permitted until 2026-07-29 10:35:24.424815695 +0000 UTC m=+395.608172108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/6b99c00e-deb7-4632-8caf-06e2ccfa05b2-certificates") pod "keda-operator-77f648d694-zgfb5" (UID: "6b99c00e-deb7-4632-8caf-06e2ccfa05b2") : references non-existent secret key: ca.crt
Jul 29 10:35:23.925246 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.925227    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/6b99c00e-deb7-4632-8caf-06e2ccfa05b2-cabundle0\") pod \"keda-operator-77f648d694-zgfb5\" (UID: \"6b99c00e-deb7-4632-8caf-06e2ccfa05b2\") " pod="openshift-keda/keda-operator-77f648d694-zgfb5"
Jul 29 10:35:23.933068 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:23.933045    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7pjx\" (UniqueName: \"kubernetes.io/projected/6b99c00e-deb7-4632-8caf-06e2ccfa05b2-kube-api-access-g7pjx\") pod \"keda-operator-77f648d694-zgfb5\" (UID: \"6b99c00e-deb7-4632-8caf-06e2ccfa05b2\") " pod="openshift-keda/keda-operator-77f648d694-zgfb5"
Jul 29 10:35:24.428476 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:24.428439    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6b99c00e-deb7-4632-8caf-06e2ccfa05b2-certificates\") pod \"keda-operator-77f648d694-zgfb5\" (UID: \"6b99c00e-deb7-4632-8caf-06e2ccfa05b2\") " pod="openshift-keda/keda-operator-77f648d694-zgfb5"
Jul 29 10:35:24.428682 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:35:24.428569    2564 secret.go:281] references non-existent secret key: ca.crt
Jul 29 10:35:24.428682 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:35:24.428583    2564 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 29 10:35:24.428682 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:35:24.428592    2564 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-zgfb5: references non-existent secret key: ca.crt
Jul 29 10:35:24.428682 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:35:24.428639    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b99c00e-deb7-4632-8caf-06e2ccfa05b2-certificates podName:6b99c00e-deb7-4632-8caf-06e2ccfa05b2 nodeName:}" failed. No retries permitted until 2026-07-29 10:35:25.428625448 +0000 UTC m=+396.611981841 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/6b99c00e-deb7-4632-8caf-06e2ccfa05b2-certificates") pod "keda-operator-77f648d694-zgfb5" (UID: "6b99c00e-deb7-4632-8caf-06e2ccfa05b2") : references non-existent secret key: ca.crt
Jul 29 10:35:25.436737 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:25.436685    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6b99c00e-deb7-4632-8caf-06e2ccfa05b2-certificates\") pod \"keda-operator-77f648d694-zgfb5\" (UID: \"6b99c00e-deb7-4632-8caf-06e2ccfa05b2\") " pod="openshift-keda/keda-operator-77f648d694-zgfb5"
Jul 29 10:35:25.439169 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:25.439148    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6b99c00e-deb7-4632-8caf-06e2ccfa05b2-certificates\") pod \"keda-operator-77f648d694-zgfb5\" (UID: \"6b99c00e-deb7-4632-8caf-06e2ccfa05b2\") " pod="openshift-keda/keda-operator-77f648d694-zgfb5"
Jul 29 10:35:25.513493 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:25.513457    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-zgfb5"
Jul 29 10:35:25.634228 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:25.634194    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-zgfb5"]
Jul 29 10:35:25.637146 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:35:25.637122    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b99c00e_deb7_4632_8caf_06e2ccfa05b2.slice/crio-2dcae17d638f9182e0719d646c12ccd0f15e0b36a5469488ca334ddadb37553f WatchSource:0}: Error finding container 2dcae17d638f9182e0719d646c12ccd0f15e0b36a5469488ca334ddadb37553f: Status 404 returned error can't find the container with id 2dcae17d638f9182e0719d646c12ccd0f15e0b36a5469488ca334ddadb37553f
Jul 29 10:35:25.786136 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:25.786097    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-zgfb5" event={"ID":"6b99c00e-deb7-4632-8caf-06e2ccfa05b2","Type":"ContainerStarted","Data":"2dcae17d638f9182e0719d646c12ccd0f15e0b36a5469488ca334ddadb37553f"}
Jul 29 10:35:28.799681 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:28.799587    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-zgfb5" event={"ID":"6b99c00e-deb7-4632-8caf-06e2ccfa05b2","Type":"ContainerStarted","Data":"2c684da6f6e881d4acc8cd5b032b8e7dbcd9a560f7048e3182482e0d5279fb3c"}
Jul 29 10:35:28.800092 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:28.799712    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-77f648d694-zgfb5"
Jul 29 10:35:28.817001 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:28.816953    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-77f648d694-zgfb5" podStartSLOduration=2.9737149 podStartE2EDuration="5.816938311s" totalImagesPullingTime="2.843223411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:35:28.815242384 +0000 UTC m=+399.998598798" watchObservedRunningTime="2026-07-29 10:35:28.816938311 +0000 UTC m=+400.000294725"
Jul 29 10:35:44.783457 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:44.783426    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-rsq4p"
Jul 29 10:35:49.809266 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:35:49.809229    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-77f648d694-zgfb5"
Jul 29 10:36:40.436097 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:40.436062    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-4b2nd"]
Jul 29 10:36:40.437995 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:40.437978    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-65f566c74b-4b2nd"
Jul 29 10:36:40.440815 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:40.440779    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 29 10:36:40.440911 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:40.440838    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jul 29 10:36:40.441895 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:40.441871    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 29 10:36:40.442000 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:40.441871    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-nlfbq\""
Jul 29 10:36:40.455257 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:40.455232    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-4b2nd"]
Jul 29 10:36:40.497888 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:40.497856    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gcp4\" (UniqueName: \"kubernetes.io/projected/d5a4d91d-38c5-41bb-9a47-dfc605a6dd99-kube-api-access-2gcp4\") pod \"kserve-controller-manager-65f566c74b-4b2nd\" (UID: \"d5a4d91d-38c5-41bb-9a47-dfc605a6dd99\") " pod="kserve/kserve-controller-manager-65f566c74b-4b2nd"
Jul 29 10:36:40.498038 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:40.497935    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5a4d91d-38c5-41bb-9a47-dfc605a6dd99-cert\") pod \"kserve-controller-manager-65f566c74b-4b2nd\" (UID: \"d5a4d91d-38c5-41bb-9a47-dfc605a6dd99\") " pod="kserve/kserve-controller-manager-65f566c74b-4b2nd"
Jul 29 10:36:40.599181 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:40.599144    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5a4d91d-38c5-41bb-9a47-dfc605a6dd99-cert\") pod \"kserve-controller-manager-65f566c74b-4b2nd\" (UID: \"d5a4d91d-38c5-41bb-9a47-dfc605a6dd99\") " pod="kserve/kserve-controller-manager-65f566c74b-4b2nd"
Jul 29 10:36:40.599181 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:40.599184    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gcp4\" (UniqueName: \"kubernetes.io/projected/d5a4d91d-38c5-41bb-9a47-dfc605a6dd99-kube-api-access-2gcp4\") pod \"kserve-controller-manager-65f566c74b-4b2nd\" (UID: \"d5a4d91d-38c5-41bb-9a47-dfc605a6dd99\") " pod="kserve/kserve-controller-manager-65f566c74b-4b2nd"
Jul 29 10:36:40.599429 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:36:40.599285    2564 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 29 10:36:40.599429 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:36:40.599349    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5a4d91d-38c5-41bb-9a47-dfc605a6dd99-cert podName:d5a4d91d-38c5-41bb-9a47-dfc605a6dd99 nodeName:}" failed. No retries permitted until 2026-07-29 10:36:41.099331659 +0000 UTC m=+472.282688056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d5a4d91d-38c5-41bb-9a47-dfc605a6dd99-cert") pod "kserve-controller-manager-65f566c74b-4b2nd" (UID: "d5a4d91d-38c5-41bb-9a47-dfc605a6dd99") : secret "kserve-webhook-server-cert" not found
Jul 29 10:36:40.614878 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:40.614843    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gcp4\" (UniqueName: \"kubernetes.io/projected/d5a4d91d-38c5-41bb-9a47-dfc605a6dd99-kube-api-access-2gcp4\") pod \"kserve-controller-manager-65f566c74b-4b2nd\" (UID: \"d5a4d91d-38c5-41bb-9a47-dfc605a6dd99\") " pod="kserve/kserve-controller-manager-65f566c74b-4b2nd"
Jul 29 10:36:41.103324 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:41.103288    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5a4d91d-38c5-41bb-9a47-dfc605a6dd99-cert\") pod \"kserve-controller-manager-65f566c74b-4b2nd\" (UID: \"d5a4d91d-38c5-41bb-9a47-dfc605a6dd99\") " pod="kserve/kserve-controller-manager-65f566c74b-4b2nd"
Jul 29 10:36:41.105608 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:41.105582    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5a4d91d-38c5-41bb-9a47-dfc605a6dd99-cert\") pod \"kserve-controller-manager-65f566c74b-4b2nd\" (UID: \"d5a4d91d-38c5-41bb-9a47-dfc605a6dd99\") " pod="kserve/kserve-controller-manager-65f566c74b-4b2nd"
Jul 29 10:36:41.347981 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:41.347943    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-65f566c74b-4b2nd"
Jul 29 10:36:41.469845 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:41.469790    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-4b2nd"]
Jul 29 10:36:41.471885 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:36:41.471860    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5a4d91d_38c5_41bb_9a47_dfc605a6dd99.slice/crio-55a858e22307f4b856735a153224f4b0844f403d04ceb9ce9941bd41db74535d WatchSource:0}: Error finding container 55a858e22307f4b856735a153224f4b0844f403d04ceb9ce9941bd41db74535d: Status 404 returned error can't find the container with id 55a858e22307f4b856735a153224f4b0844f403d04ceb9ce9941bd41db74535d
Jul 29 10:36:42.059253 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:42.059203    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-65f566c74b-4b2nd" event={"ID":"d5a4d91d-38c5-41bb-9a47-dfc605a6dd99","Type":"ContainerStarted","Data":"55a858e22307f4b856735a153224f4b0844f403d04ceb9ce9941bd41db74535d"}
Jul 29 10:36:44.069248 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:44.069206    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-65f566c74b-4b2nd" event={"ID":"d5a4d91d-38c5-41bb-9a47-dfc605a6dd99","Type":"ContainerStarted","Data":"02fdb6b4a8a14240cc62736a18a020e90d88d4e1a4914d21695c5bb7c59a84ae"}
Jul 29 10:36:44.069607 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:44.069313    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-65f566c74b-4b2nd"
Jul 29 10:36:44.084752 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:36:44.084702    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-65f566c74b-4b2nd" podStartSLOduration=1.660585503 podStartE2EDuration="4.084688553s" totalImagesPullingTime="2.42410305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:36:44.08386295 +0000 UTC m=+475.267219366" watchObservedRunningTime="2026-07-29 10:36:44.084688553 +0000 UTC m=+475.268044967"
Jul 29 10:37:15.077992 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:15.077961    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-65f566c74b-4b2nd"
Jul 29 10:37:23.369742 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.369708    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-4b2nd"]
Jul 29 10:37:23.370213 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.369901    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-65f566c74b-4b2nd" podUID="d5a4d91d-38c5-41bb-9a47-dfc605a6dd99" containerName="manager" containerID="cri-o://02fdb6b4a8a14240cc62736a18a020e90d88d4e1a4914d21695c5bb7c59a84ae" gracePeriod=10
Jul 29 10:37:23.394759 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.394732    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-vtfxb"]
Jul 29 10:37:23.396592 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.396577    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-65f566c74b-vtfxb"
Jul 29 10:37:23.406013 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.405990    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-vtfxb"]
Jul 29 10:37:23.515983 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.515945    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqhvh\" (UniqueName: \"kubernetes.io/projected/aeecb00a-3e07-473e-848f-24261afa6a4a-kube-api-access-nqhvh\") pod \"kserve-controller-manager-65f566c74b-vtfxb\" (UID: \"aeecb00a-3e07-473e-848f-24261afa6a4a\") " pod="kserve/kserve-controller-manager-65f566c74b-vtfxb"
Jul 29 10:37:23.515983 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.515996    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aeecb00a-3e07-473e-848f-24261afa6a4a-cert\") pod \"kserve-controller-manager-65f566c74b-vtfxb\" (UID: \"aeecb00a-3e07-473e-848f-24261afa6a4a\") " pod="kserve/kserve-controller-manager-65f566c74b-vtfxb"
Jul 29 10:37:23.600916 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.600891    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-65f566c74b-4b2nd"
Jul 29 10:37:23.616684 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.616659    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aeecb00a-3e07-473e-848f-24261afa6a4a-cert\") pod \"kserve-controller-manager-65f566c74b-vtfxb\" (UID: \"aeecb00a-3e07-473e-848f-24261afa6a4a\") " pod="kserve/kserve-controller-manager-65f566c74b-vtfxb"
Jul 29 10:37:23.616818 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.616724    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqhvh\" (UniqueName: \"kubernetes.io/projected/aeecb00a-3e07-473e-848f-24261afa6a4a-kube-api-access-nqhvh\") pod \"kserve-controller-manager-65f566c74b-vtfxb\" (UID: \"aeecb00a-3e07-473e-848f-24261afa6a4a\") " pod="kserve/kserve-controller-manager-65f566c74b-vtfxb"
Jul 29 10:37:23.619740 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.619704    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aeecb00a-3e07-473e-848f-24261afa6a4a-cert\") pod \"kserve-controller-manager-65f566c74b-vtfxb\" (UID: \"aeecb00a-3e07-473e-848f-24261afa6a4a\") " pod="kserve/kserve-controller-manager-65f566c74b-vtfxb"
Jul 29 10:37:23.627839 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.627728    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqhvh\" (UniqueName: \"kubernetes.io/projected/aeecb00a-3e07-473e-848f-24261afa6a4a-kube-api-access-nqhvh\") pod \"kserve-controller-manager-65f566c74b-vtfxb\" (UID: \"aeecb00a-3e07-473e-848f-24261afa6a4a\") " pod="kserve/kserve-controller-manager-65f566c74b-vtfxb"
Jul 29 10:37:23.717777 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.717743    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gcp4\" (UniqueName: \"kubernetes.io/projected/d5a4d91d-38c5-41bb-9a47-dfc605a6dd99-kube-api-access-2gcp4\") pod \"d5a4d91d-38c5-41bb-9a47-dfc605a6dd99\" (UID: \"d5a4d91d-38c5-41bb-9a47-dfc605a6dd99\") "
Jul 29 10:37:23.717988 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.717787    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5a4d91d-38c5-41bb-9a47-dfc605a6dd99-cert\") pod \"d5a4d91d-38c5-41bb-9a47-dfc605a6dd99\" (UID: \"d5a4d91d-38c5-41bb-9a47-dfc605a6dd99\") "
Jul 29 10:37:23.719949 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.719915    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5a4d91d-38c5-41bb-9a47-dfc605a6dd99-cert" (OuterVolumeSpecName: "cert") pod "d5a4d91d-38c5-41bb-9a47-dfc605a6dd99" (UID: "d5a4d91d-38c5-41bb-9a47-dfc605a6dd99"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:37:23.719949 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.719938    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5a4d91d-38c5-41bb-9a47-dfc605a6dd99-kube-api-access-2gcp4" (OuterVolumeSpecName: "kube-api-access-2gcp4") pod "d5a4d91d-38c5-41bb-9a47-dfc605a6dd99" (UID: "d5a4d91d-38c5-41bb-9a47-dfc605a6dd99"). InnerVolumeSpecName "kube-api-access-2gcp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:37:23.742239 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.742218    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-65f566c74b-vtfxb"
Jul 29 10:37:23.819414 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.819357    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gcp4\" (UniqueName: \"kubernetes.io/projected/d5a4d91d-38c5-41bb-9a47-dfc605a6dd99-kube-api-access-2gcp4\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:37:23.819414 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.819397    2564 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5a4d91d-38c5-41bb-9a47-dfc605a6dd99-cert\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:37:23.863751 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:23.863728    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-vtfxb"]
Jul 29 10:37:23.865915 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:37:23.865889    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeecb00a_3e07_473e_848f_24261afa6a4a.slice/crio-1ef15927274083f11168af6048af858fdff26f8ba6dec460a8382b4d82faac26 WatchSource:0}: Error finding container 1ef15927274083f11168af6048af858fdff26f8ba6dec460a8382b4d82faac26: Status 404 returned error can't find the container with id 1ef15927274083f11168af6048af858fdff26f8ba6dec460a8382b4d82faac26
Jul 29 10:37:24.212435 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:24.212401    2564 generic.go:356] "Generic (PLEG): container finished" podID="d5a4d91d-38c5-41bb-9a47-dfc605a6dd99" containerID="02fdb6b4a8a14240cc62736a18a020e90d88d4e1a4914d21695c5bb7c59a84ae" exitCode=0
Jul 29 10:37:24.212602 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:24.212442    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-65f566c74b-4b2nd"
Jul 29 10:37:24.212602 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:24.212489    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-65f566c74b-4b2nd" event={"ID":"d5a4d91d-38c5-41bb-9a47-dfc605a6dd99","Type":"ContainerDied","Data":"02fdb6b4a8a14240cc62736a18a020e90d88d4e1a4914d21695c5bb7c59a84ae"}
Jul 29 10:37:24.212602 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:24.212544    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-65f566c74b-4b2nd" event={"ID":"d5a4d91d-38c5-41bb-9a47-dfc605a6dd99","Type":"ContainerDied","Data":"55a858e22307f4b856735a153224f4b0844f403d04ceb9ce9941bd41db74535d"}
Jul 29 10:37:24.212602 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:24.212568    2564 scope.go:117] "RemoveContainer" containerID="02fdb6b4a8a14240cc62736a18a020e90d88d4e1a4914d21695c5bb7c59a84ae"
Jul 29 10:37:24.213660 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:24.213641    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-65f566c74b-vtfxb" event={"ID":"aeecb00a-3e07-473e-848f-24261afa6a4a","Type":"ContainerStarted","Data":"1ef15927274083f11168af6048af858fdff26f8ba6dec460a8382b4d82faac26"}
Jul 29 10:37:24.220896 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:24.220872    2564 scope.go:117] "RemoveContainer" containerID="02fdb6b4a8a14240cc62736a18a020e90d88d4e1a4914d21695c5bb7c59a84ae"
Jul 29 10:37:24.221133 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:37:24.221114    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02fdb6b4a8a14240cc62736a18a020e90d88d4e1a4914d21695c5bb7c59a84ae\": container with ID starting with 02fdb6b4a8a14240cc62736a18a020e90d88d4e1a4914d21695c5bb7c59a84ae not found: ID does not exist" containerID="02fdb6b4a8a14240cc62736a18a020e90d88d4e1a4914d21695c5bb7c59a84ae"
Jul 29 10:37:24.221178 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:24.221142    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02fdb6b4a8a14240cc62736a18a020e90d88d4e1a4914d21695c5bb7c59a84ae"} err="failed to get container status \"02fdb6b4a8a14240cc62736a18a020e90d88d4e1a4914d21695c5bb7c59a84ae\": rpc error: code = NotFound desc = could not find container \"02fdb6b4a8a14240cc62736a18a020e90d88d4e1a4914d21695c5bb7c59a84ae\": container with ID starting with 02fdb6b4a8a14240cc62736a18a020e90d88d4e1a4914d21695c5bb7c59a84ae not found: ID does not exist"
Jul 29 10:37:24.233409 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:24.233387    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-4b2nd"]
Jul 29 10:37:24.237212 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:24.237193    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-4b2nd"]
Jul 29 10:37:25.218176 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:25.218140    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-65f566c74b-vtfxb" event={"ID":"aeecb00a-3e07-473e-848f-24261afa6a4a","Type":"ContainerStarted","Data":"b62c2cacb5f2365ba20f14e55c804522afc04d0a4754ad5cb8cb4eaff0b515fa"}
Jul 29 10:37:25.218618 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:25.218207    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-65f566c74b-vtfxb"
Jul 29 10:37:25.235613 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:25.235567    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-65f566c74b-vtfxb" podStartSLOduration=1.846221305 podStartE2EDuration="2.235554413s" totalImagesPullingTime="389.333108ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:37:25.233629044 +0000 UTC m=+516.416985474" watchObservedRunningTime="2026-07-29 10:37:25.235554413 +0000 UTC m=+516.418910827"
Jul 29 10:37:25.342352 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:25.342323    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5a4d91d-38c5-41bb-9a47-dfc605a6dd99" path="/var/lib/kubelet/pods/d5a4d91d-38c5-41bb-9a47-dfc605a6dd99/volumes"
Jul 29 10:37:56.228979 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:37:56.228952    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-65f566c74b-vtfxb"
Jul 29 10:43:07.815742 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:07.815703    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"]
Jul 29 10:43:07.816281 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:07.815996    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5a4d91d-38c5-41bb-9a47-dfc605a6dd99" containerName="manager"
Jul 29 10:43:07.816281 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:07.816008    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5a4d91d-38c5-41bb-9a47-dfc605a6dd99" containerName="manager"
Jul 29 10:43:07.816281 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:07.816067    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5a4d91d-38c5-41bb-9a47-dfc605a6dd99" containerName="manager"
Jul 29 10:43:07.819094 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:07.819070    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"
Jul 29 10:43:07.821982 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:07.821954    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-877ef-serving-cert\""
Jul 29 10:43:07.822126 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:07.821955    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-877ef-kube-rbac-proxy-sar-config\""
Jul 29 10:43:07.822126 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:07.822051    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 29 10:43:07.822224 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:07.822192    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-6nnxs\""
Jul 29 10:43:07.829419 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:07.829376    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"]
Jul 29 10:43:07.961223 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:07.961188    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e3f2a347-74b3-4418-9aea-1da0a051bcb8-proxy-tls\") pod \"model-chainer-raw-877ef-79fd4bcc7f-sdbr2\" (UID: \"e3f2a347-74b3-4418-9aea-1da0a051bcb8\") " pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"
Jul 29 10:43:07.961412 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:07.961307    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3f2a347-74b3-4418-9aea-1da0a051bcb8-openshift-service-ca-bundle\") pod \"model-chainer-raw-877ef-79fd4bcc7f-sdbr2\" (UID: \"e3f2a347-74b3-4418-9aea-1da0a051bcb8\") " pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"
Jul 29 10:43:08.062201 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:08.062164    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3f2a347-74b3-4418-9aea-1da0a051bcb8-openshift-service-ca-bundle\") pod \"model-chainer-raw-877ef-79fd4bcc7f-sdbr2\" (UID: \"e3f2a347-74b3-4418-9aea-1da0a051bcb8\") " pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"
Jul 29 10:43:08.062417 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:08.062227    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e3f2a347-74b3-4418-9aea-1da0a051bcb8-proxy-tls\") pod \"model-chainer-raw-877ef-79fd4bcc7f-sdbr2\" (UID: \"e3f2a347-74b3-4418-9aea-1da0a051bcb8\") " pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"
Jul 29 10:43:08.062822 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:08.062786    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3f2a347-74b3-4418-9aea-1da0a051bcb8-openshift-service-ca-bundle\") pod \"model-chainer-raw-877ef-79fd4bcc7f-sdbr2\" (UID: \"e3f2a347-74b3-4418-9aea-1da0a051bcb8\") " pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"
Jul 29 10:43:08.064551 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:08.064529    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e3f2a347-74b3-4418-9aea-1da0a051bcb8-proxy-tls\") pod \"model-chainer-raw-877ef-79fd4bcc7f-sdbr2\" (UID: \"e3f2a347-74b3-4418-9aea-1da0a051bcb8\") " pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"
Jul 29 10:43:08.130284 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:08.130200    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"
Jul 29 10:43:08.257193 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:08.257115    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"]
Jul 29 10:43:08.259849 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:43:08.259827    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3f2a347_74b3_4418_9aea_1da0a051bcb8.slice/crio-65849ec6552cebc3aac5a9d8510f2512164e9dfc96ceae04548a1b69f475e801 WatchSource:0}: Error finding container 65849ec6552cebc3aac5a9d8510f2512164e9dfc96ceae04548a1b69f475e801: Status 404 returned error can't find the container with id 65849ec6552cebc3aac5a9d8510f2512164e9dfc96ceae04548a1b69f475e801
Jul 29 10:43:08.261697 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:08.261679    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 10:43:08.423126 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:08.423052    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2" event={"ID":"e3f2a347-74b3-4418-9aea-1da0a051bcb8","Type":"ContainerStarted","Data":"65849ec6552cebc3aac5a9d8510f2512164e9dfc96ceae04548a1b69f475e801"}
Jul 29 10:43:10.430867 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:10.430826    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2" event={"ID":"e3f2a347-74b3-4418-9aea-1da0a051bcb8","Type":"ContainerStarted","Data":"6a8e517c0d8b94c65dd85522851e12eaa3e516e20e1f0e37e1a111b7718abac3"}
Jul 29 10:43:10.431258 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:10.430933    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"
Jul 29 10:43:10.448457 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:10.448412    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2" podStartSLOduration=1.4797779260000001 podStartE2EDuration="3.448398138s" totalImagesPullingTime="1.968620212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:43:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:43:10.447604078 +0000 UTC m=+861.630960493" watchObservedRunningTime="2026-07-29 10:43:10.448398138 +0000 UTC m=+861.631754578"
Jul 29 10:43:16.441922 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:16.441891    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"
Jul 29 10:43:17.852400 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:17.852364    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"]
Jul 29 10:43:17.852811 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:17.852610    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2" podUID="e3f2a347-74b3-4418-9aea-1da0a051bcb8" containerName="model-chainer-raw-877ef" containerID="cri-o://6a8e517c0d8b94c65dd85522851e12eaa3e516e20e1f0e37e1a111b7718abac3" gracePeriod=30
Jul 29 10:43:21.438817 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:21.438762    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2" podUID="e3f2a347-74b3-4418-9aea-1da0a051bcb8" containerName="model-chainer-raw-877ef" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 10:43:26.438954 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:26.438910    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2" podUID="e3f2a347-74b3-4418-9aea-1da0a051bcb8" containerName="model-chainer-raw-877ef" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 10:43:31.438815 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:31.438765    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2" podUID="e3f2a347-74b3-4418-9aea-1da0a051bcb8" containerName="model-chainer-raw-877ef" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 10:43:31.439250 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:31.438887    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"
Jul 29 10:43:36.438905 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:36.438868    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2" podUID="e3f2a347-74b3-4418-9aea-1da0a051bcb8" containerName="model-chainer-raw-877ef" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 10:43:41.438509 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:41.438468    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2" podUID="e3f2a347-74b3-4418-9aea-1da0a051bcb8" containerName="model-chainer-raw-877ef" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 10:43:46.438587 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:46.438548    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2" podUID="e3f2a347-74b3-4418-9aea-1da0a051bcb8" containerName="model-chainer-raw-877ef" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 10:43:47.992401 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:47.992378    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"
Jul 29 10:43:48.049626 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:48.049579    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e3f2a347-74b3-4418-9aea-1da0a051bcb8-proxy-tls\") pod \"e3f2a347-74b3-4418-9aea-1da0a051bcb8\" (UID: \"e3f2a347-74b3-4418-9aea-1da0a051bcb8\") "
Jul 29 10:43:48.049626 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:48.049628    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3f2a347-74b3-4418-9aea-1da0a051bcb8-openshift-service-ca-bundle\") pod \"e3f2a347-74b3-4418-9aea-1da0a051bcb8\" (UID: \"e3f2a347-74b3-4418-9aea-1da0a051bcb8\") "
Jul 29 10:43:48.050107 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:48.050076    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3f2a347-74b3-4418-9aea-1da0a051bcb8-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "e3f2a347-74b3-4418-9aea-1da0a051bcb8" (UID: "e3f2a347-74b3-4418-9aea-1da0a051bcb8"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:43:48.051820 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:48.051778    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3f2a347-74b3-4418-9aea-1da0a051bcb8-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e3f2a347-74b3-4418-9aea-1da0a051bcb8" (UID: "e3f2a347-74b3-4418-9aea-1da0a051bcb8"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:43:48.150931 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:48.150840    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e3f2a347-74b3-4418-9aea-1da0a051bcb8-proxy-tls\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:43:48.150931 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:48.150880    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3f2a347-74b3-4418-9aea-1da0a051bcb8-openshift-service-ca-bundle\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:43:48.566038 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:48.566007    2564 generic.go:356] "Generic (PLEG): container finished" podID="e3f2a347-74b3-4418-9aea-1da0a051bcb8" containerID="6a8e517c0d8b94c65dd85522851e12eaa3e516e20e1f0e37e1a111b7718abac3" exitCode=0
Jul 29 10:43:48.566213 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:48.566051    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"
Jul 29 10:43:48.566213 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:48.566086    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2" event={"ID":"e3f2a347-74b3-4418-9aea-1da0a051bcb8","Type":"ContainerDied","Data":"6a8e517c0d8b94c65dd85522851e12eaa3e516e20e1f0e37e1a111b7718abac3"}
Jul 29 10:43:48.566213 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:48.566126    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2" event={"ID":"e3f2a347-74b3-4418-9aea-1da0a051bcb8","Type":"ContainerDied","Data":"65849ec6552cebc3aac5a9d8510f2512164e9dfc96ceae04548a1b69f475e801"}
Jul 29 10:43:48.566213 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:48.566147    2564 scope.go:117] "RemoveContainer" containerID="6a8e517c0d8b94c65dd85522851e12eaa3e516e20e1f0e37e1a111b7718abac3"
Jul 29 10:43:48.574706 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:48.574686    2564 scope.go:117] "RemoveContainer" containerID="6a8e517c0d8b94c65dd85522851e12eaa3e516e20e1f0e37e1a111b7718abac3"
Jul 29 10:43:48.575047 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:43:48.575017    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a8e517c0d8b94c65dd85522851e12eaa3e516e20e1f0e37e1a111b7718abac3\": container with ID starting with 6a8e517c0d8b94c65dd85522851e12eaa3e516e20e1f0e37e1a111b7718abac3 not found: ID does not exist" containerID="6a8e517c0d8b94c65dd85522851e12eaa3e516e20e1f0e37e1a111b7718abac3"
Jul 29 10:43:48.575110 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:48.575057    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a8e517c0d8b94c65dd85522851e12eaa3e516e20e1f0e37e1a111b7718abac3"} err="failed to get container status \"6a8e517c0d8b94c65dd85522851e12eaa3e516e20e1f0e37e1a111b7718abac3\": rpc error: code = NotFound desc = could not find container \"6a8e517c0d8b94c65dd85522851e12eaa3e516e20e1f0e37e1a111b7718abac3\": container with ID starting with 6a8e517c0d8b94c65dd85522851e12eaa3e516e20e1f0e37e1a111b7718abac3 not found: ID does not exist"
Jul 29 10:43:48.587407 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:48.587377    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"]
Jul 29 10:43:48.591642 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:48.591619    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-877ef-79fd4bcc7f-sdbr2"]
Jul 29 10:43:49.342256 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:43:49.342228    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3f2a347-74b3-4418-9aea-1da0a051bcb8" path="/var/lib/kubelet/pods/e3f2a347-74b3-4418-9aea-1da0a051bcb8/volumes"
Jul 29 10:44:58.124300 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.124261    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"]
Jul 29 10:44:58.127771 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.125338    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3f2a347-74b3-4418-9aea-1da0a051bcb8" containerName="model-chainer-raw-877ef"
Jul 29 10:44:58.127771 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.125364    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3f2a347-74b3-4418-9aea-1da0a051bcb8" containerName="model-chainer-raw-877ef"
Jul 29 10:44:58.127771 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.125479    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3f2a347-74b3-4418-9aea-1da0a051bcb8" containerName="model-chainer-raw-877ef"
Jul 29 10:44:58.129122 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.129104    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"
Jul 29 10:44:58.131634 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.131595    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-2d850-serving-cert\""
Jul 29 10:44:58.131745 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.131638    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-6nnxs\""
Jul 29 10:44:58.131745 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.131647    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-2d850-kube-rbac-proxy-sar-config\""
Jul 29 10:44:58.131745 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.131638    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 29 10:44:58.136137 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.136075    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"]
Jul 29 10:44:58.172320 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.172287    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acefd807-184c-40b3-a1c5-7a9154a1153f-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-2d850-6b684bf475-4fgjt\" (UID: \"acefd807-184c-40b3-a1c5-7a9154a1153f\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"
Jul 29 10:44:58.172460 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.172327    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/acefd807-184c-40b3-a1c5-7a9154a1153f-proxy-tls\") pod \"model-chainer-raw-hpa-2d850-6b684bf475-4fgjt\" (UID: \"acefd807-184c-40b3-a1c5-7a9154a1153f\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"
Jul 29 10:44:58.273473 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.273439    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acefd807-184c-40b3-a1c5-7a9154a1153f-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-2d850-6b684bf475-4fgjt\" (UID: \"acefd807-184c-40b3-a1c5-7a9154a1153f\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"
Jul 29 10:44:58.273643 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.273483    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/acefd807-184c-40b3-a1c5-7a9154a1153f-proxy-tls\") pod \"model-chainer-raw-hpa-2d850-6b684bf475-4fgjt\" (UID: \"acefd807-184c-40b3-a1c5-7a9154a1153f\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"
Jul 29 10:44:58.274106 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.274084    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acefd807-184c-40b3-a1c5-7a9154a1153f-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-2d850-6b684bf475-4fgjt\" (UID: \"acefd807-184c-40b3-a1c5-7a9154a1153f\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"
Jul 29 10:44:58.275812 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.275778    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/acefd807-184c-40b3-a1c5-7a9154a1153f-proxy-tls\") pod \"model-chainer-raw-hpa-2d850-6b684bf475-4fgjt\" (UID: \"acefd807-184c-40b3-a1c5-7a9154a1153f\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"
Jul 29 10:44:58.439232 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.439116    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"
Jul 29 10:44:58.563950 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.563791    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"]
Jul 29 10:44:58.566482 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:44:58.566448    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacefd807_184c_40b3_a1c5_7a9154a1153f.slice/crio-5a1d754e59a95db057ee52945b8f51fc748b37e3b970bff9865dc5699b9bb12d WatchSource:0}: Error finding container 5a1d754e59a95db057ee52945b8f51fc748b37e3b970bff9865dc5699b9bb12d: Status 404 returned error can't find the container with id 5a1d754e59a95db057ee52945b8f51fc748b37e3b970bff9865dc5699b9bb12d
Jul 29 10:44:58.812939 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.812906    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt" event={"ID":"acefd807-184c-40b3-a1c5-7a9154a1153f","Type":"ContainerStarted","Data":"2f3b373831f9e6f3adbd8b4c5e8abcdf41fe2d37e07a7e5741ad7fae62f07580"}
Jul 29 10:44:58.813097 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.812955    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt" event={"ID":"acefd807-184c-40b3-a1c5-7a9154a1153f","Type":"ContainerStarted","Data":"5a1d754e59a95db057ee52945b8f51fc748b37e3b970bff9865dc5699b9bb12d"}
Jul 29 10:44:58.813097 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.812987    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"
Jul 29 10:44:58.828448 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:44:58.828395    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt" podStartSLOduration=0.82837771 podStartE2EDuration="828.37771ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:44:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:44:58.82810256 +0000 UTC m=+970.011458974" watchObservedRunningTime="2026-07-29 10:44:58.82837771 +0000 UTC m=+970.011734128"
Jul 29 10:45:04.822087 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:04.822054    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"
Jul 29 10:45:08.183251 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:08.183220    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"]
Jul 29 10:45:08.183679 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:08.183469    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt" podUID="acefd807-184c-40b3-a1c5-7a9154a1153f" containerName="model-chainer-raw-hpa-2d850" containerID="cri-o://2f3b373831f9e6f3adbd8b4c5e8abcdf41fe2d37e07a7e5741ad7fae62f07580" gracePeriod=30
Jul 29 10:45:09.821361 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:09.821316    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt" podUID="acefd807-184c-40b3-a1c5-7a9154a1153f" containerName="model-chainer-raw-hpa-2d850" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 10:45:14.820692 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:14.820609    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt" podUID="acefd807-184c-40b3-a1c5-7a9154a1153f" containerName="model-chainer-raw-hpa-2d850" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 10:45:19.821457 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:19.821412    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt" podUID="acefd807-184c-40b3-a1c5-7a9154a1153f" containerName="model-chainer-raw-hpa-2d850" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 10:45:19.821852 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:19.821520    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"
Jul 29 10:45:24.821414 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:24.821370    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt" podUID="acefd807-184c-40b3-a1c5-7a9154a1153f" containerName="model-chainer-raw-hpa-2d850" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 10:45:29.820995 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:29.820953    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt" podUID="acefd807-184c-40b3-a1c5-7a9154a1153f" containerName="model-chainer-raw-hpa-2d850" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 10:45:34.821545 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:34.821508    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt" podUID="acefd807-184c-40b3-a1c5-7a9154a1153f" containerName="model-chainer-raw-hpa-2d850" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 10:45:38.828620 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:38.828596    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"
Jul 29 10:45:38.854189 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:38.854155    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acefd807-184c-40b3-a1c5-7a9154a1153f-openshift-service-ca-bundle\") pod \"acefd807-184c-40b3-a1c5-7a9154a1153f\" (UID: \"acefd807-184c-40b3-a1c5-7a9154a1153f\") "
Jul 29 10:45:38.854312 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:38.854224    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/acefd807-184c-40b3-a1c5-7a9154a1153f-proxy-tls\") pod \"acefd807-184c-40b3-a1c5-7a9154a1153f\" (UID: \"acefd807-184c-40b3-a1c5-7a9154a1153f\") "
Jul 29 10:45:38.854536 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:38.854511    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acefd807-184c-40b3-a1c5-7a9154a1153f-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "acefd807-184c-40b3-a1c5-7a9154a1153f" (UID: "acefd807-184c-40b3-a1c5-7a9154a1153f"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:45:38.856186 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:38.856161    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acefd807-184c-40b3-a1c5-7a9154a1153f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "acefd807-184c-40b3-a1c5-7a9154a1153f" (UID: "acefd807-184c-40b3-a1c5-7a9154a1153f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:45:38.955795 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:38.955710    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/acefd807-184c-40b3-a1c5-7a9154a1153f-proxy-tls\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:45:38.955795 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:38.955745    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acefd807-184c-40b3-a1c5-7a9154a1153f-openshift-service-ca-bundle\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:45:38.959501 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:38.959473    2564 generic.go:356] "Generic (PLEG): container finished" podID="acefd807-184c-40b3-a1c5-7a9154a1153f" containerID="2f3b373831f9e6f3adbd8b4c5e8abcdf41fe2d37e07a7e5741ad7fae62f07580" exitCode=0
Jul 29 10:45:38.959635 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:38.959520    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"
Jul 29 10:45:38.959635 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:38.959545    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt" event={"ID":"acefd807-184c-40b3-a1c5-7a9154a1153f","Type":"ContainerDied","Data":"2f3b373831f9e6f3adbd8b4c5e8abcdf41fe2d37e07a7e5741ad7fae62f07580"}
Jul 29 10:45:38.959635 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:38.959585    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt" event={"ID":"acefd807-184c-40b3-a1c5-7a9154a1153f","Type":"ContainerDied","Data":"5a1d754e59a95db057ee52945b8f51fc748b37e3b970bff9865dc5699b9bb12d"}
Jul 29 10:45:38.959635 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:38.959607    2564 scope.go:117] "RemoveContainer" containerID="2f3b373831f9e6f3adbd8b4c5e8abcdf41fe2d37e07a7e5741ad7fae62f07580"
Jul 29 10:45:38.967985 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:38.967960    2564 scope.go:117] "RemoveContainer" containerID="2f3b373831f9e6f3adbd8b4c5e8abcdf41fe2d37e07a7e5741ad7fae62f07580"
Jul 29 10:45:38.968225 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:45:38.968208    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f3b373831f9e6f3adbd8b4c5e8abcdf41fe2d37e07a7e5741ad7fae62f07580\": container with ID starting with 2f3b373831f9e6f3adbd8b4c5e8abcdf41fe2d37e07a7e5741ad7fae62f07580 not found: ID does not exist" containerID="2f3b373831f9e6f3adbd8b4c5e8abcdf41fe2d37e07a7e5741ad7fae62f07580"
Jul 29 10:45:38.968275 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:38.968236    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f3b373831f9e6f3adbd8b4c5e8abcdf41fe2d37e07a7e5741ad7fae62f07580"} err="failed to get container status \"2f3b373831f9e6f3adbd8b4c5e8abcdf41fe2d37e07a7e5741ad7fae62f07580\": rpc error: code = NotFound desc = could not find container \"2f3b373831f9e6f3adbd8b4c5e8abcdf41fe2d37e07a7e5741ad7fae62f07580\": container with ID starting with 2f3b373831f9e6f3adbd8b4c5e8abcdf41fe2d37e07a7e5741ad7fae62f07580 not found: ID does not exist"
Jul 29 10:45:38.981444 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:38.981418    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"]
Jul 29 10:45:38.985227 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:38.985206    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-2d850-6b684bf475-4fgjt"]
Jul 29 10:45:39.341968 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:45:39.341937    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acefd807-184c-40b3-a1c5-7a9154a1153f" path="/var/lib/kubelet/pods/acefd807-184c-40b3-a1c5-7a9154a1153f/volumes"
Jul 29 10:53:04.500584 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.500508    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"]
Jul 29 10:53:04.501044 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.500768    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acefd807-184c-40b3-a1c5-7a9154a1153f" containerName="model-chainer-raw-hpa-2d850"
Jul 29 10:53:04.501044 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.500779    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="acefd807-184c-40b3-a1c5-7a9154a1153f" containerName="model-chainer-raw-hpa-2d850"
Jul 29 10:53:04.501044 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.500843    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="acefd807-184c-40b3-a1c5-7a9154a1153f" containerName="model-chainer-raw-hpa-2d850"
Jul 29 10:53:04.503870 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.503838    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.506303 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.506282    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-6nnxs\""
Jul 29 10:53:04.506851 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.506831    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 29 10:53:04.506851 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.506848    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 29 10:53:04.507021 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.506862    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\""
Jul 29 10:53:04.507489 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.507470    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 29 10:53:04.507612 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.507526    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-v2-predictor-serving-cert\""
Jul 29 10:53:04.507612 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.507554    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 29 10:53:04.514299 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.514279    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"]
Jul 29 10:53:04.595254 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.595225    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.595424 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.595268    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxbmn\" (UniqueName: \"kubernetes.io/projected/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kube-api-access-dxbmn\") pod \"isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.595424 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.595288    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.595424 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.595341    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.595424 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.595406    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.696842 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.696776    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.697125 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.697105    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxbmn\" (UniqueName: \"kubernetes.io/projected/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kube-api-access-dxbmn\") pod \"isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.697235 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.697221    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.697829 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.697333    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.697829 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.697382    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.701355 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.700787    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.701355 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.701274    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.701557 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.701527    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.702148 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.702129    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.710211 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.710188    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxbmn\" (UniqueName: \"kubernetes.io/projected/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kube-api-access-dxbmn\") pod \"isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.814268 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.814182    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:04.942025 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.941999    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"]
Jul 29 10:53:04.944694 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:53:04.944666    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f43cc27_6aab_4fd6_bb18_301b4885c8a0.slice/crio-ac6c629a6b5c74d3c707a3f32dbb017fd5c70308cd9c9a80a6a4522b6d89139e WatchSource:0}: Error finding container ac6c629a6b5c74d3c707a3f32dbb017fd5c70308cd9c9a80a6a4522b6d89139e: Status 404 returned error can't find the container with id ac6c629a6b5c74d3c707a3f32dbb017fd5c70308cd9c9a80a6a4522b6d89139e
Jul 29 10:53:04.946691 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:04.946674    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 10:53:05.510362 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:05.510328    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" event={"ID":"3f43cc27-6aab-4fd6-bb18-301b4885c8a0","Type":"ContainerStarted","Data":"ac6c629a6b5c74d3c707a3f32dbb017fd5c70308cd9c9a80a6a4522b6d89139e"}
Jul 29 10:53:09.529129 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:09.529090    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" event={"ID":"3f43cc27-6aab-4fd6-bb18-301b4885c8a0","Type":"ContainerStarted","Data":"f37bea58f152a00481aa63e1747ddfeda91acae4504305a325fc07b6968423a9"}
Jul 29 10:53:12.541495 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:12.541458    2564 generic.go:356] "Generic (PLEG): container finished" podID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerID="f37bea58f152a00481aa63e1747ddfeda91acae4504305a325fc07b6968423a9" exitCode=0
Jul 29 10:53:12.541854 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:12.541527    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" event={"ID":"3f43cc27-6aab-4fd6-bb18-301b4885c8a0","Type":"ContainerDied","Data":"f37bea58f152a00481aa63e1747ddfeda91acae4504305a325fc07b6968423a9"}
Jul 29 10:53:25.602341 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:25.602303    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" event={"ID":"3f43cc27-6aab-4fd6-bb18-301b4885c8a0","Type":"ContainerStarted","Data":"b266e740aa7c3fe6e5acdd59f5d778842d48f04fc1dd4b971eb0f9ef3567c7a7"}
Jul 29 10:53:28.618183 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:28.618144    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" event={"ID":"3f43cc27-6aab-4fd6-bb18-301b4885c8a0","Type":"ContainerStarted","Data":"442d5a22890ebcd63a3157d93684b4f3a5cf83deec930d472defa8d4cc844e20"}
Jul 29 10:53:28.618590 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:28.618208    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:28.618590 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:28.618247    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:28.619622 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:28.619600    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 29 10:53:28.637400 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:28.637334    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" podStartSLOduration=1.8404874979999999 podStartE2EDuration="24.637316868s" totalImagesPullingTime="19.011754666s" totalInitContainerRuntime="3.785074704s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:53:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:53:28.636638473 +0000 UTC m=+1479.819994912" watchObservedRunningTime="2026-07-29 10:53:28.637316868 +0000 UTC m=+1479.820673285"
Jul 29 10:53:29.622062 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:29.622022    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 29 10:53:34.627053 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:34.627021    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:53:34.627630 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:34.627604    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 29 10:53:44.628257 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:44.628217    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 29 10:53:54.627852 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:53:54.627793    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 29 10:54:04.627528 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:04.627489    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 29 10:54:14.628088 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:14.628003    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 29 10:54:24.627913 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:24.627874    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 29 10:54:34.628505 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:34.628458    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 29 10:54:44.628317 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:44.628279    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:54:54.597191 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:54.597158    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"]
Jul 29 10:54:54.597567 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:54.597393    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kserve-container" containerID="cri-o://b266e740aa7c3fe6e5acdd59f5d778842d48f04fc1dd4b971eb0f9ef3567c7a7" gracePeriod=30
Jul 29 10:54:54.597567 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:54.597493    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kube-rbac-proxy" containerID="cri-o://442d5a22890ebcd63a3157d93684b4f3a5cf83deec930d472defa8d4cc844e20" gracePeriod=30
Jul 29 10:54:54.623099 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:54.623068    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.22:8643/healthz\": dial tcp 10.133.0.22:8643: connect: connection refused"
Jul 29 10:54:54.925899 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:54.925792    2564 generic.go:356] "Generic (PLEG): container finished" podID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerID="442d5a22890ebcd63a3157d93684b4f3a5cf83deec930d472defa8d4cc844e20" exitCode=2
Jul 29 10:54:54.925899 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:54.925866    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" event={"ID":"3f43cc27-6aab-4fd6-bb18-301b4885c8a0","Type":"ContainerDied","Data":"442d5a22890ebcd63a3157d93684b4f3a5cf83deec930d472defa8d4cc844e20"}
Jul 29 10:54:59.232983 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.232960    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:54:59.281080 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.280995    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxbmn\" (UniqueName: \"kubernetes.io/projected/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kube-api-access-dxbmn\") pod \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") "
Jul 29 10:54:59.281080 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.281051    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-cabundle-cert\") pod \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") "
Jul 29 10:54:59.281080 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.281078    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kube-rbac-proxy-sar-config\") pod \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") "
Jul 29 10:54:59.281343 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.281101    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kserve-provision-location\") pod \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") "
Jul 29 10:54:59.281343 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.281117    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-proxy-tls\") pod \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\" (UID: \"3f43cc27-6aab-4fd6-bb18-301b4885c8a0\") "
Jul 29 10:54:59.281452 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.281427    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3f43cc27-6aab-4fd6-bb18-301b4885c8a0" (UID: "3f43cc27-6aab-4fd6-bb18-301b4885c8a0"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:54:59.281495 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.281441    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "3f43cc27-6aab-4fd6-bb18-301b4885c8a0" (UID: "3f43cc27-6aab-4fd6-bb18-301b4885c8a0"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 10:54:59.281495 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.281468    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "3f43cc27-6aab-4fd6-bb18-301b4885c8a0" (UID: "3f43cc27-6aab-4fd6-bb18-301b4885c8a0"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:54:59.283205 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.283168    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kube-api-access-dxbmn" (OuterVolumeSpecName: "kube-api-access-dxbmn") pod "3f43cc27-6aab-4fd6-bb18-301b4885c8a0" (UID: "3f43cc27-6aab-4fd6-bb18-301b4885c8a0"). InnerVolumeSpecName "kube-api-access-dxbmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:54:59.283205 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.283180    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3f43cc27-6aab-4fd6-bb18-301b4885c8a0" (UID: "3f43cc27-6aab-4fd6-bb18-301b4885c8a0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:54:59.382069 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.382032    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxbmn\" (UniqueName: \"kubernetes.io/projected/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kube-api-access-dxbmn\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:54:59.382069 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.382062    2564 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-cabundle-cert\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:54:59.382069 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.382075    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:54:59.382296 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.382086    2564 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-kserve-provision-location\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:54:59.382296 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.382098    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f43cc27-6aab-4fd6-bb18-301b4885c8a0-proxy-tls\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:54:59.774727 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.774685    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"]
Jul 29 10:54:59.774979 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.774966    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="storage-initializer"
Jul 29 10:54:59.775022 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.774979    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="storage-initializer"
Jul 29 10:54:59.775022 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.774991    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kube-rbac-proxy"
Jul 29 10:54:59.775022 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.774996    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kube-rbac-proxy"
Jul 29 10:54:59.775022 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.775007    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kserve-container"
Jul 29 10:54:59.775022 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.775012    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kserve-container"
Jul 29 10:54:59.775164 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.775053    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kube-rbac-proxy"
Jul 29 10:54:59.775164 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.775062    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerName="kserve-container"
Jul 29 10:54:59.778087 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.778063    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:54:59.780472 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.780448    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-predictor-serving-cert\""
Jul 29 10:54:59.780565 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.780449    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\""
Jul 29 10:54:59.786235 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.786210    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-79b59b5966-m9887\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:54:59.786333 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.786261    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-79b59b5966-m9887\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:54:59.786333 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.786321    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebb3977e-c145-4398-a89e-87dee4f6bf7a-proxy-tls\") pod \"isvc-canary-rollback-predictor-79b59b5966-m9887\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:54:59.786418 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.786350    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trnp6\" (UniqueName: \"kubernetes.io/projected/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kube-api-access-trnp6\") pod \"isvc-canary-rollback-predictor-79b59b5966-m9887\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:54:59.786418 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.786377    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebb3977e-c145-4398-a89e-87dee4f6bf7a-cabundle-cert\") pod \"isvc-canary-rollback-predictor-79b59b5966-m9887\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:54:59.793917 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.793895    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"]
Jul 29 10:54:59.887231 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.887184    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebb3977e-c145-4398-a89e-87dee4f6bf7a-cabundle-cert\") pod \"isvc-canary-rollback-predictor-79b59b5966-m9887\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:54:59.887387 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.887286    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-79b59b5966-m9887\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:54:59.887387 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.887327    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-79b59b5966-m9887\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:54:59.887387 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.887363    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebb3977e-c145-4398-a89e-87dee4f6bf7a-proxy-tls\") pod \"isvc-canary-rollback-predictor-79b59b5966-m9887\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:54:59.887683 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.887394    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trnp6\" (UniqueName: \"kubernetes.io/projected/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kube-api-access-trnp6\") pod \"isvc-canary-rollback-predictor-79b59b5966-m9887\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:54:59.887837 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.887793    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-79b59b5966-m9887\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:54:59.887955 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.887936    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebb3977e-c145-4398-a89e-87dee4f6bf7a-cabundle-cert\") pod \"isvc-canary-rollback-predictor-79b59b5966-m9887\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:54:59.888080 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.888062    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-79b59b5966-m9887\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:54:59.890002 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.889981    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebb3977e-c145-4398-a89e-87dee4f6bf7a-proxy-tls\") pod \"isvc-canary-rollback-predictor-79b59b5966-m9887\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:54:59.895433 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.895411    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trnp6\" (UniqueName: \"kubernetes.io/projected/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kube-api-access-trnp6\") pod \"isvc-canary-rollback-predictor-79b59b5966-m9887\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:54:59.946330 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.946296    2564 generic.go:356] "Generic (PLEG): container finished" podID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" containerID="b266e740aa7c3fe6e5acdd59f5d778842d48f04fc1dd4b971eb0f9ef3567c7a7" exitCode=0
Jul 29 10:54:59.946480 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.946357    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"
Jul 29 10:54:59.946480 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.946375    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" event={"ID":"3f43cc27-6aab-4fd6-bb18-301b4885c8a0","Type":"ContainerDied","Data":"b266e740aa7c3fe6e5acdd59f5d778842d48f04fc1dd4b971eb0f9ef3567c7a7"}
Jul 29 10:54:59.946480 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.946436    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl" event={"ID":"3f43cc27-6aab-4fd6-bb18-301b4885c8a0","Type":"ContainerDied","Data":"ac6c629a6b5c74d3c707a3f32dbb017fd5c70308cd9c9a80a6a4522b6d89139e"}
Jul 29 10:54:59.946480 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.946465    2564 scope.go:117] "RemoveContainer" containerID="442d5a22890ebcd63a3157d93684b4f3a5cf83deec930d472defa8d4cc844e20"
Jul 29 10:54:59.954533 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.954516    2564 scope.go:117] "RemoveContainer" containerID="b266e740aa7c3fe6e5acdd59f5d778842d48f04fc1dd4b971eb0f9ef3567c7a7"
Jul 29 10:54:59.961734 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.961713    2564 scope.go:117] "RemoveContainer" containerID="f37bea58f152a00481aa63e1747ddfeda91acae4504305a325fc07b6968423a9"
Jul 29 10:54:59.963035 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.963015    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"]
Jul 29 10:54:59.970041 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.970015    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-869c8cbf4f-78zgl"]
Jul 29 10:54:59.970261 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.970242    2564 scope.go:117] "RemoveContainer" containerID="442d5a22890ebcd63a3157d93684b4f3a5cf83deec930d472defa8d4cc844e20"
Jul 29 10:54:59.972198 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:54:59.972097    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"442d5a22890ebcd63a3157d93684b4f3a5cf83deec930d472defa8d4cc844e20\": container with ID starting with 442d5a22890ebcd63a3157d93684b4f3a5cf83deec930d472defa8d4cc844e20 not found: ID does not exist" containerID="442d5a22890ebcd63a3157d93684b4f3a5cf83deec930d472defa8d4cc844e20"
Jul 29 10:54:59.972198 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.972143    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"442d5a22890ebcd63a3157d93684b4f3a5cf83deec930d472defa8d4cc844e20"} err="failed to get container status \"442d5a22890ebcd63a3157d93684b4f3a5cf83deec930d472defa8d4cc844e20\": rpc error: code = NotFound desc = could not find container \"442d5a22890ebcd63a3157d93684b4f3a5cf83deec930d472defa8d4cc844e20\": container with ID starting with 442d5a22890ebcd63a3157d93684b4f3a5cf83deec930d472defa8d4cc844e20 not found: ID does not exist"
Jul 29 10:54:59.972198 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.972192    2564 scope.go:117] "RemoveContainer" containerID="b266e740aa7c3fe6e5acdd59f5d778842d48f04fc1dd4b971eb0f9ef3567c7a7"
Jul 29 10:54:59.973245 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:54:59.972641    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b266e740aa7c3fe6e5acdd59f5d778842d48f04fc1dd4b971eb0f9ef3567c7a7\": container with ID starting with b266e740aa7c3fe6e5acdd59f5d778842d48f04fc1dd4b971eb0f9ef3567c7a7 not found: ID does not exist" containerID="b266e740aa7c3fe6e5acdd59f5d778842d48f04fc1dd4b971eb0f9ef3567c7a7"
Jul 29 10:54:59.973245 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.972681    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b266e740aa7c3fe6e5acdd59f5d778842d48f04fc1dd4b971eb0f9ef3567c7a7"} err="failed to get container status \"b266e740aa7c3fe6e5acdd59f5d778842d48f04fc1dd4b971eb0f9ef3567c7a7\": rpc error: code = NotFound desc = could not find container \"b266e740aa7c3fe6e5acdd59f5d778842d48f04fc1dd4b971eb0f9ef3567c7a7\": container with ID starting with b266e740aa7c3fe6e5acdd59f5d778842d48f04fc1dd4b971eb0f9ef3567c7a7 not found: ID does not exist"
Jul 29 10:54:59.973245 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.972704    2564 scope.go:117] "RemoveContainer" containerID="f37bea58f152a00481aa63e1747ddfeda91acae4504305a325fc07b6968423a9"
Jul 29 10:54:59.973540 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:54:59.973515    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f37bea58f152a00481aa63e1747ddfeda91acae4504305a325fc07b6968423a9\": container with ID starting with f37bea58f152a00481aa63e1747ddfeda91acae4504305a325fc07b6968423a9 not found: ID does not exist" containerID="f37bea58f152a00481aa63e1747ddfeda91acae4504305a325fc07b6968423a9"
Jul 29 10:54:59.973773 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:54:59.973553    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f37bea58f152a00481aa63e1747ddfeda91acae4504305a325fc07b6968423a9"} err="failed to get container status \"f37bea58f152a00481aa63e1747ddfeda91acae4504305a325fc07b6968423a9\": rpc error: code = NotFound desc = could not find container \"f37bea58f152a00481aa63e1747ddfeda91acae4504305a325fc07b6968423a9\": container with ID starting with f37bea58f152a00481aa63e1747ddfeda91acae4504305a325fc07b6968423a9 not found: ID does not exist"
Jul 29 10:55:00.087654 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:00.087567    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:55:00.218005 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:00.217929    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"]
Jul 29 10:55:00.220295 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:55:00.220271    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebb3977e_c145_4398_a89e_87dee4f6bf7a.slice/crio-2c75cfb78ac776baf8b1ce0e90d04a895bbc39de63a41c1a07690730563166dd WatchSource:0}: Error finding container 2c75cfb78ac776baf8b1ce0e90d04a895bbc39de63a41c1a07690730563166dd: Status 404 returned error can't find the container with id 2c75cfb78ac776baf8b1ce0e90d04a895bbc39de63a41c1a07690730563166dd
Jul 29 10:55:00.952428 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:00.952392    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" event={"ID":"ebb3977e-c145-4398-a89e-87dee4f6bf7a","Type":"ContainerStarted","Data":"2916cce589472d3a23e7582f5886f7c166a66ee31959af14f5925e63d7d24b11"}
Jul 29 10:55:00.952889 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:00.952450    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" event={"ID":"ebb3977e-c145-4398-a89e-87dee4f6bf7a","Type":"ContainerStarted","Data":"2c75cfb78ac776baf8b1ce0e90d04a895bbc39de63a41c1a07690730563166dd"}
Jul 29 10:55:01.342717 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:01.342687    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f43cc27-6aab-4fd6-bb18-301b4885c8a0" path="/var/lib/kubelet/pods/3f43cc27-6aab-4fd6-bb18-301b4885c8a0/volumes"
Jul 29 10:55:03.966407 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:03.966374    2564 generic.go:356] "Generic (PLEG): container finished" podID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerID="2916cce589472d3a23e7582f5886f7c166a66ee31959af14f5925e63d7d24b11" exitCode=0
Jul 29 10:55:03.966894 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:03.966450    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" event={"ID":"ebb3977e-c145-4398-a89e-87dee4f6bf7a","Type":"ContainerDied","Data":"2916cce589472d3a23e7582f5886f7c166a66ee31959af14f5925e63d7d24b11"}
Jul 29 10:55:04.971955 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:04.971916    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" event={"ID":"ebb3977e-c145-4398-a89e-87dee4f6bf7a","Type":"ContainerStarted","Data":"d29b7479236b1985607e17db6e3b9fde1cea69147892cca4806d06aa833fb514"}
Jul 29 10:55:04.972451 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:04.971992    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" event={"ID":"ebb3977e-c145-4398-a89e-87dee4f6bf7a","Type":"ContainerStarted","Data":"a6ca9f4b32bf6ef9b1abde54931c81330e79e2a6bd0a599341d6d1d3fd04406f"}
Jul 29 10:55:04.972451 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:04.972213    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:55:04.992585 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:04.992539    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" podStartSLOduration=2.354105548 podStartE2EDuration="5.992523545s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.638417997s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:54:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:55:04.990650068 +0000 UTC m=+1576.174006488" watchObservedRunningTime="2026-07-29 10:55:04.992523545 +0000 UTC m=+1576.175879960"
Jul 29 10:55:05.975406 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:05.975364    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:55:05.976757 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:05.976729    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 29 10:55:06.978636 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:06.978597    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 29 10:55:11.983144 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:11.983114    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:55:11.983629 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:11.983603    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 29 10:55:21.983780 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:21.983739    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 29 10:55:31.984565 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:31.984523    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 29 10:55:41.984019 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:41.983931    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 29 10:55:51.984025 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:55:51.983984    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 29 10:56:01.984515 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:56:01.984474    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 29 10:56:11.983621 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:56:11.983578    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 29 10:56:21.983996 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:56:21.983962    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:58:00.065611 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:00.065576    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"]
Jul 29 10:58:00.066183 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:00.065830    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kserve-container" containerID="cri-o://a6ca9f4b32bf6ef9b1abde54931c81330e79e2a6bd0a599341d6d1d3fd04406f" gracePeriod=30
Jul 29 10:58:00.066183 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:00.065893    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kube-rbac-proxy" containerID="cri-o://d29b7479236b1985607e17db6e3b9fde1cea69147892cca4806d06aa833fb514" gracePeriod=30
Jul 29 10:58:00.596913 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:00.596875    2564 generic.go:356] "Generic (PLEG): container finished" podID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerID="d29b7479236b1985607e17db6e3b9fde1cea69147892cca4806d06aa833fb514" exitCode=2
Jul 29 10:58:00.597090 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:00.596923    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" event={"ID":"ebb3977e-c145-4398-a89e-87dee4f6bf7a","Type":"ContainerDied","Data":"d29b7479236b1985607e17db6e3b9fde1cea69147892cca4806d06aa833fb514"}
Jul 29 10:58:01.979256 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:01.979223    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.23:8643/healthz\": dial tcp 10.133.0.23:8643: connect: connection refused"
Jul 29 10:58:01.983535 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:01.983507    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 29 10:58:04.610072 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.610048    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:58:04.613288 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.613265    2564 generic.go:356] "Generic (PLEG): container finished" podID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerID="a6ca9f4b32bf6ef9b1abde54931c81330e79e2a6bd0a599341d6d1d3fd04406f" exitCode=0
Jul 29 10:58:04.613409 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.613328    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"
Jul 29 10:58:04.613409 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.613343    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" event={"ID":"ebb3977e-c145-4398-a89e-87dee4f6bf7a","Type":"ContainerDied","Data":"a6ca9f4b32bf6ef9b1abde54931c81330e79e2a6bd0a599341d6d1d3fd04406f"}
Jul 29 10:58:04.613409 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.613401    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887" event={"ID":"ebb3977e-c145-4398-a89e-87dee4f6bf7a","Type":"ContainerDied","Data":"2c75cfb78ac776baf8b1ce0e90d04a895bbc39de63a41c1a07690730563166dd"}
Jul 29 10:58:04.613561 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.613430    2564 scope.go:117] "RemoveContainer" containerID="d29b7479236b1985607e17db6e3b9fde1cea69147892cca4806d06aa833fb514"
Jul 29 10:58:04.620630 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.620614    2564 scope.go:117] "RemoveContainer" containerID="a6ca9f4b32bf6ef9b1abde54931c81330e79e2a6bd0a599341d6d1d3fd04406f"
Jul 29 10:58:04.627520 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.627499    2564 scope.go:117] "RemoveContainer" containerID="2916cce589472d3a23e7582f5886f7c166a66ee31959af14f5925e63d7d24b11"
Jul 29 10:58:04.635893 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.635874    2564 scope.go:117] "RemoveContainer" containerID="d29b7479236b1985607e17db6e3b9fde1cea69147892cca4806d06aa833fb514"
Jul 29 10:58:04.636147 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:58:04.636127    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d29b7479236b1985607e17db6e3b9fde1cea69147892cca4806d06aa833fb514\": container with ID starting with d29b7479236b1985607e17db6e3b9fde1cea69147892cca4806d06aa833fb514 not found: ID does not exist" containerID="d29b7479236b1985607e17db6e3b9fde1cea69147892cca4806d06aa833fb514"
Jul 29 10:58:04.636215 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.636160    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d29b7479236b1985607e17db6e3b9fde1cea69147892cca4806d06aa833fb514"} err="failed to get container status \"d29b7479236b1985607e17db6e3b9fde1cea69147892cca4806d06aa833fb514\": rpc error: code = NotFound desc = could not find container \"d29b7479236b1985607e17db6e3b9fde1cea69147892cca4806d06aa833fb514\": container with ID starting with d29b7479236b1985607e17db6e3b9fde1cea69147892cca4806d06aa833fb514 not found: ID does not exist"
Jul 29 10:58:04.636215 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.636187    2564 scope.go:117] "RemoveContainer" containerID="a6ca9f4b32bf6ef9b1abde54931c81330e79e2a6bd0a599341d6d1d3fd04406f"
Jul 29 10:58:04.636440 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:58:04.636420    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6ca9f4b32bf6ef9b1abde54931c81330e79e2a6bd0a599341d6d1d3fd04406f\": container with ID starting with a6ca9f4b32bf6ef9b1abde54931c81330e79e2a6bd0a599341d6d1d3fd04406f not found: ID does not exist" containerID="a6ca9f4b32bf6ef9b1abde54931c81330e79e2a6bd0a599341d6d1d3fd04406f"
Jul 29 10:58:04.636492 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.636446    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6ca9f4b32bf6ef9b1abde54931c81330e79e2a6bd0a599341d6d1d3fd04406f"} err="failed to get container status \"a6ca9f4b32bf6ef9b1abde54931c81330e79e2a6bd0a599341d6d1d3fd04406f\": rpc error: code = NotFound desc = could not find container \"a6ca9f4b32bf6ef9b1abde54931c81330e79e2a6bd0a599341d6d1d3fd04406f\": container with ID starting with a6ca9f4b32bf6ef9b1abde54931c81330e79e2a6bd0a599341d6d1d3fd04406f not found: ID does not exist"
Jul 29 10:58:04.636492 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.636462    2564 scope.go:117] "RemoveContainer" containerID="2916cce589472d3a23e7582f5886f7c166a66ee31959af14f5925e63d7d24b11"
Jul 29 10:58:04.636684 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:58:04.636666    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2916cce589472d3a23e7582f5886f7c166a66ee31959af14f5925e63d7d24b11\": container with ID starting with 2916cce589472d3a23e7582f5886f7c166a66ee31959af14f5925e63d7d24b11 not found: ID does not exist" containerID="2916cce589472d3a23e7582f5886f7c166a66ee31959af14f5925e63d7d24b11"
Jul 29 10:58:04.636716 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.636692    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2916cce589472d3a23e7582f5886f7c166a66ee31959af14f5925e63d7d24b11"} err="failed to get container status \"2916cce589472d3a23e7582f5886f7c166a66ee31959af14f5925e63d7d24b11\": rpc error: code = NotFound desc = could not find container \"2916cce589472d3a23e7582f5886f7c166a66ee31959af14f5925e63d7d24b11\": container with ID starting with 2916cce589472d3a23e7582f5886f7c166a66ee31959af14f5925e63d7d24b11 not found: ID does not exist"
Jul 29 10:58:04.697210 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.697130    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trnp6\" (UniqueName: \"kubernetes.io/projected/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kube-api-access-trnp6\") pod \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") "
Jul 29 10:58:04.697210 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.697168    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kserve-provision-location\") pod \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") "
Jul 29 10:58:04.697210 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.697203    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kube-rbac-proxy-sar-config\") pod \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") "
Jul 29 10:58:04.697528 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.697250    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebb3977e-c145-4398-a89e-87dee4f6bf7a-cabundle-cert\") pod \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") "
Jul 29 10:58:04.697528 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.697281    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebb3977e-c145-4398-a89e-87dee4f6bf7a-proxy-tls\") pod \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\" (UID: \"ebb3977e-c145-4398-a89e-87dee4f6bf7a\") "
Jul 29 10:58:04.697635 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.697475    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "ebb3977e-c145-4398-a89e-87dee4f6bf7a" (UID: "ebb3977e-c145-4398-a89e-87dee4f6bf7a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 10:58:04.697692 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.697630    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebb3977e-c145-4398-a89e-87dee4f6bf7a-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "ebb3977e-c145-4398-a89e-87dee4f6bf7a" (UID: "ebb3977e-c145-4398-a89e-87dee4f6bf7a"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:58:04.697692 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.697664    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ebb3977e-c145-4398-a89e-87dee4f6bf7a" (UID: "ebb3977e-c145-4398-a89e-87dee4f6bf7a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:58:04.699308 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.699284    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebb3977e-c145-4398-a89e-87dee4f6bf7a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ebb3977e-c145-4398-a89e-87dee4f6bf7a" (UID: "ebb3977e-c145-4398-a89e-87dee4f6bf7a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:58:04.699400 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.699367    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kube-api-access-trnp6" (OuterVolumeSpecName: "kube-api-access-trnp6") pod "ebb3977e-c145-4398-a89e-87dee4f6bf7a" (UID: "ebb3977e-c145-4398-a89e-87dee4f6bf7a"). InnerVolumeSpecName "kube-api-access-trnp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:58:04.798622 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.798584    2564 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebb3977e-c145-4398-a89e-87dee4f6bf7a-cabundle-cert\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:58:04.798622 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.798618    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebb3977e-c145-4398-a89e-87dee4f6bf7a-proxy-tls\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:58:04.798622 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.798627    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trnp6\" (UniqueName: \"kubernetes.io/projected/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kube-api-access-trnp6\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:58:04.798622 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.798636    2564 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kserve-provision-location\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:58:04.798870 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.798646    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebb3977e-c145-4398-a89e-87dee4f6bf7a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:58:04.936207 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.936178    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"]
Jul 29 10:58:04.939826 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:04.939788    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-79b59b5966-m9887"]
Jul 29 10:58:05.174932 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.174896    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"]
Jul 29 10:58:05.175181 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.175162    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kserve-container"
Jul 29 10:58:05.175244 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.175183    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kserve-container"
Jul 29 10:58:05.175244 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.175203    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kube-rbac-proxy"
Jul 29 10:58:05.175244 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.175208    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kube-rbac-proxy"
Jul 29 10:58:05.175244 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.175215    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="storage-initializer"
Jul 29 10:58:05.175244 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.175220    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="storage-initializer"
Jul 29 10:58:05.175429 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.175268    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kube-rbac-proxy"
Jul 29 10:58:05.175429 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.175275    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" containerName="kserve-container"
Jul 29 10:58:05.179547 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.179526    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.182339 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.182311    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 29 10:58:05.182545 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.182529    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-6nnxs\""
Jul 29 10:58:05.182608 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.182588    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 29 10:58:05.182655 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.182611    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 29 10:58:05.182765 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.182750    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-stop-kube-rbac-proxy-sar-config\""
Jul 29 10:58:05.182854 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.182842    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-stop-v2-predictor-serving-cert\""
Jul 29 10:58:05.183098 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.183084    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 29 10:58:05.190700 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.190681    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"]
Jul 29 10:58:05.201726 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.201636    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4db18cb9-fe2f-4241-be77-49364295ac96-cabundle-cert\") pod \"isvc-canary-stop-v2-predictor-865fc6b775-gs95d\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.201726 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.201683    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4db18cb9-fe2f-4241-be77-49364295ac96-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-v2-predictor-865fc6b775-gs95d\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.201964 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.201747    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4db18cb9-fe2f-4241-be77-49364295ac96-proxy-tls\") pod \"isvc-canary-stop-v2-predictor-865fc6b775-gs95d\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.201964 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.201772    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdlx2\" (UniqueName: \"kubernetes.io/projected/4db18cb9-fe2f-4241-be77-49364295ac96-kube-api-access-cdlx2\") pod \"isvc-canary-stop-v2-predictor-865fc6b775-gs95d\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.201964 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.201839    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4db18cb9-fe2f-4241-be77-49364295ac96-kserve-provision-location\") pod \"isvc-canary-stop-v2-predictor-865fc6b775-gs95d\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.303105 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.303072    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4db18cb9-fe2f-4241-be77-49364295ac96-proxy-tls\") pod \"isvc-canary-stop-v2-predictor-865fc6b775-gs95d\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.303105 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.303105    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdlx2\" (UniqueName: \"kubernetes.io/projected/4db18cb9-fe2f-4241-be77-49364295ac96-kube-api-access-cdlx2\") pod \"isvc-canary-stop-v2-predictor-865fc6b775-gs95d\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.303312 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.303138    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4db18cb9-fe2f-4241-be77-49364295ac96-kserve-provision-location\") pod \"isvc-canary-stop-v2-predictor-865fc6b775-gs95d\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.303312 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.303175    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4db18cb9-fe2f-4241-be77-49364295ac96-cabundle-cert\") pod \"isvc-canary-stop-v2-predictor-865fc6b775-gs95d\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.303312 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.303192    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4db18cb9-fe2f-4241-be77-49364295ac96-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-v2-predictor-865fc6b775-gs95d\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.303632 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.303611    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4db18cb9-fe2f-4241-be77-49364295ac96-kserve-provision-location\") pod \"isvc-canary-stop-v2-predictor-865fc6b775-gs95d\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.303915 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.303895    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4db18cb9-fe2f-4241-be77-49364295ac96-cabundle-cert\") pod \"isvc-canary-stop-v2-predictor-865fc6b775-gs95d\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.303992 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.303974    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4db18cb9-fe2f-4241-be77-49364295ac96-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-v2-predictor-865fc6b775-gs95d\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.305544 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.305527    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4db18cb9-fe2f-4241-be77-49364295ac96-proxy-tls\") pod \"isvc-canary-stop-v2-predictor-865fc6b775-gs95d\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.311749 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.311725    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdlx2\" (UniqueName: \"kubernetes.io/projected/4db18cb9-fe2f-4241-be77-49364295ac96-kube-api-access-cdlx2\") pod \"isvc-canary-stop-v2-predictor-865fc6b775-gs95d\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") " pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.343141 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.343111    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebb3977e-c145-4398-a89e-87dee4f6bf7a" path="/var/lib/kubelet/pods/ebb3977e-c145-4398-a89e-87dee4f6bf7a/volumes"
Jul 29 10:58:05.490844 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.490790    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:05.621771 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:05.621743    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"]
Jul 29 10:58:05.624893 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:58:05.624853    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4db18cb9_fe2f_4241_be77_49364295ac96.slice/crio-8e8f9fcf24bc989a07200d47da1ffcf64c773198fdf364e0b45551a50ec7a868 WatchSource:0}: Error finding container 8e8f9fcf24bc989a07200d47da1ffcf64c773198fdf364e0b45551a50ec7a868: Status 404 returned error can't find the container with id 8e8f9fcf24bc989a07200d47da1ffcf64c773198fdf364e0b45551a50ec7a868
Jul 29 10:58:06.622525 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:06.622489    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" event={"ID":"4db18cb9-fe2f-4241-be77-49364295ac96","Type":"ContainerStarted","Data":"bf8c64780949e0300b214c8026ef8a90a4dd7e41319ce4f17ba0be1d14cae06a"}
Jul 29 10:58:06.622909 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:06.622574    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" event={"ID":"4db18cb9-fe2f-4241-be77-49364295ac96","Type":"ContainerStarted","Data":"8e8f9fcf24bc989a07200d47da1ffcf64c773198fdf364e0b45551a50ec7a868"}
Jul 29 10:58:09.634209 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:09.634177    2564 generic.go:356] "Generic (PLEG): container finished" podID="4db18cb9-fe2f-4241-be77-49364295ac96" containerID="bf8c64780949e0300b214c8026ef8a90a4dd7e41319ce4f17ba0be1d14cae06a" exitCode=0
Jul 29 10:58:09.634550 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:09.634260    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" event={"ID":"4db18cb9-fe2f-4241-be77-49364295ac96","Type":"ContainerDied","Data":"bf8c64780949e0300b214c8026ef8a90a4dd7e41319ce4f17ba0be1d14cae06a"}
Jul 29 10:58:10.640385 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:10.640349    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" event={"ID":"4db18cb9-fe2f-4241-be77-49364295ac96","Type":"ContainerStarted","Data":"f60216deef1ab34c76f1ed42cb7b0354d97192015390ae1c1b9bca90e7441798"}
Jul 29 10:58:10.640862 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:10.640411    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" event={"ID":"4db18cb9-fe2f-4241-be77-49364295ac96","Type":"ContainerStarted","Data":"4c738af7c8a2bf8c268c85ce4ee63415f3283e30607866516aa5cce48f91b1a5"}
Jul 29 10:58:10.640862 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:10.640589    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:10.660874 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:10.660825    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" podStartSLOduration=1.775831421 podStartE2EDuration="5.660792105s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.884960684s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:58:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:58:10.659069971 +0000 UTC m=+1761.842426387" watchObservedRunningTime="2026-07-29 10:58:10.660792105 +0000 UTC m=+1761.844148585"
Jul 29 10:58:11.644617 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:11.644583    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:11.645625 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:11.645599    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 29 10:58:12.648603 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:12.648560    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 29 10:58:17.652734 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:17.652703    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:58:17.653204 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:17.653179    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 29 10:58:27.653920 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:27.653884    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 29 10:58:37.653707 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:37.653666    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 29 10:58:47.654050 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:47.654010    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 29 10:58:57.653222 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:58:57.653174    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 29 10:59:07.653301 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:07.653259    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 29 10:59:17.653422 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:17.653386    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 29 10:59:19.339533 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:19.339496    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 29 10:59:29.343371 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:29.343345    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:59:35.231947 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:35.231912    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"]
Jul 29 10:59:35.232348 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:35.232172    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kserve-container" containerID="cri-o://4c738af7c8a2bf8c268c85ce4ee63415f3283e30607866516aa5cce48f91b1a5" gracePeriod=30
Jul 29 10:59:35.232348 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:35.232232    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kube-rbac-proxy" containerID="cri-o://f60216deef1ab34c76f1ed42cb7b0354d97192015390ae1c1b9bca90e7441798" gracePeriod=30
Jul 29 10:59:35.947989 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:35.947955    2564 generic.go:356] "Generic (PLEG): container finished" podID="4db18cb9-fe2f-4241-be77-49364295ac96" containerID="f60216deef1ab34c76f1ed42cb7b0354d97192015390ae1c1b9bca90e7441798" exitCode=2
Jul 29 10:59:35.948158 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:35.948002    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" event={"ID":"4db18cb9-fe2f-4241-be77-49364295ac96","Type":"ContainerDied","Data":"f60216deef1ab34c76f1ed42cb7b0354d97192015390ae1c1b9bca90e7441798"}
Jul 29 10:59:37.649406 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:37.649364    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused"
Jul 29 10:59:39.339547 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.339510    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 29 10:59:39.769122 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.769099    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:59:39.824341 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.824312    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4db18cb9-fe2f-4241-be77-49364295ac96-kserve-provision-location\") pod \"4db18cb9-fe2f-4241-be77-49364295ac96\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") "
Jul 29 10:59:39.824515 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.824352    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4db18cb9-fe2f-4241-be77-49364295ac96-kube-rbac-proxy-sar-config\") pod \"4db18cb9-fe2f-4241-be77-49364295ac96\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") "
Jul 29 10:59:39.824515 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.824371    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4db18cb9-fe2f-4241-be77-49364295ac96-proxy-tls\") pod \"4db18cb9-fe2f-4241-be77-49364295ac96\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") "
Jul 29 10:59:39.824515 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.824405    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4db18cb9-fe2f-4241-be77-49364295ac96-cabundle-cert\") pod \"4db18cb9-fe2f-4241-be77-49364295ac96\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") "
Jul 29 10:59:39.824515 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.824441    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdlx2\" (UniqueName: \"kubernetes.io/projected/4db18cb9-fe2f-4241-be77-49364295ac96-kube-api-access-cdlx2\") pod \"4db18cb9-fe2f-4241-be77-49364295ac96\" (UID: \"4db18cb9-fe2f-4241-be77-49364295ac96\") "
Jul 29 10:59:39.824760 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.824695    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4db18cb9-fe2f-4241-be77-49364295ac96-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4db18cb9-fe2f-4241-be77-49364295ac96" (UID: "4db18cb9-fe2f-4241-be77-49364295ac96"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 10:59:39.824760 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.824718    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4db18cb9-fe2f-4241-be77-49364295ac96-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4db18cb9-fe2f-4241-be77-49364295ac96" (UID: "4db18cb9-fe2f-4241-be77-49364295ac96"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:59:39.824856 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.824784    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4db18cb9-fe2f-4241-be77-49364295ac96-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "4db18cb9-fe2f-4241-be77-49364295ac96" (UID: "4db18cb9-fe2f-4241-be77-49364295ac96"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:59:39.826435 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.826413    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4db18cb9-fe2f-4241-be77-49364295ac96-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4db18cb9-fe2f-4241-be77-49364295ac96" (UID: "4db18cb9-fe2f-4241-be77-49364295ac96"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:59:39.826508 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.826487    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4db18cb9-fe2f-4241-be77-49364295ac96-kube-api-access-cdlx2" (OuterVolumeSpecName: "kube-api-access-cdlx2") pod "4db18cb9-fe2f-4241-be77-49364295ac96" (UID: "4db18cb9-fe2f-4241-be77-49364295ac96"). InnerVolumeSpecName "kube-api-access-cdlx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:59:39.925134 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.925057    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdlx2\" (UniqueName: \"kubernetes.io/projected/4db18cb9-fe2f-4241-be77-49364295ac96-kube-api-access-cdlx2\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:59:39.925134 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.925086    2564 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4db18cb9-fe2f-4241-be77-49364295ac96-kserve-provision-location\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:59:39.925134 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.925100    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4db18cb9-fe2f-4241-be77-49364295ac96-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:59:39.925134 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.925112    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4db18cb9-fe2f-4241-be77-49364295ac96-proxy-tls\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:59:39.925134 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.925125    2564 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4db18cb9-fe2f-4241-be77-49364295ac96-cabundle-cert\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 10:59:39.963796 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.963762    2564 generic.go:356] "Generic (PLEG): container finished" podID="4db18cb9-fe2f-4241-be77-49364295ac96" containerID="4c738af7c8a2bf8c268c85ce4ee63415f3283e30607866516aa5cce48f91b1a5" exitCode=0
Jul 29 10:59:39.963944 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.963840    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" event={"ID":"4db18cb9-fe2f-4241-be77-49364295ac96","Type":"ContainerDied","Data":"4c738af7c8a2bf8c268c85ce4ee63415f3283e30607866516aa5cce48f91b1a5"}
Jul 29 10:59:39.963944 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.963898    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d" event={"ID":"4db18cb9-fe2f-4241-be77-49364295ac96","Type":"ContainerDied","Data":"8e8f9fcf24bc989a07200d47da1ffcf64c773198fdf364e0b45551a50ec7a868"}
Jul 29 10:59:39.963944 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.963926    2564 scope.go:117] "RemoveContainer" containerID="f60216deef1ab34c76f1ed42cb7b0354d97192015390ae1c1b9bca90e7441798"
Jul 29 10:59:39.963944 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.963848    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"
Jul 29 10:59:39.971951 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.971931    2564 scope.go:117] "RemoveContainer" containerID="4c738af7c8a2bf8c268c85ce4ee63415f3283e30607866516aa5cce48f91b1a5"
Jul 29 10:59:39.979356 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.979340    2564 scope.go:117] "RemoveContainer" containerID="bf8c64780949e0300b214c8026ef8a90a4dd7e41319ce4f17ba0be1d14cae06a"
Jul 29 10:59:39.986270 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.986225    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"]
Jul 29 10:59:39.986341 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.986289    2564 scope.go:117] "RemoveContainer" containerID="f60216deef1ab34c76f1ed42cb7b0354d97192015390ae1c1b9bca90e7441798"
Jul 29 10:59:39.986579 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:59:39.986549    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f60216deef1ab34c76f1ed42cb7b0354d97192015390ae1c1b9bca90e7441798\": container with ID starting with f60216deef1ab34c76f1ed42cb7b0354d97192015390ae1c1b9bca90e7441798 not found: ID does not exist" containerID="f60216deef1ab34c76f1ed42cb7b0354d97192015390ae1c1b9bca90e7441798"
Jul 29 10:59:39.986627 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.986586    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f60216deef1ab34c76f1ed42cb7b0354d97192015390ae1c1b9bca90e7441798"} err="failed to get container status \"f60216deef1ab34c76f1ed42cb7b0354d97192015390ae1c1b9bca90e7441798\": rpc error: code = NotFound desc = could not find container \"f60216deef1ab34c76f1ed42cb7b0354d97192015390ae1c1b9bca90e7441798\": container with ID starting with f60216deef1ab34c76f1ed42cb7b0354d97192015390ae1c1b9bca90e7441798 not found: ID does not exist"
Jul 29 10:59:39.986627 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.986605    2564 scope.go:117] "RemoveContainer" containerID="4c738af7c8a2bf8c268c85ce4ee63415f3283e30607866516aa5cce48f91b1a5"
Jul 29 10:59:39.986841 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:59:39.986823    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c738af7c8a2bf8c268c85ce4ee63415f3283e30607866516aa5cce48f91b1a5\": container with ID starting with 4c738af7c8a2bf8c268c85ce4ee63415f3283e30607866516aa5cce48f91b1a5 not found: ID does not exist" containerID="4c738af7c8a2bf8c268c85ce4ee63415f3283e30607866516aa5cce48f91b1a5"
Jul 29 10:59:39.986887 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.986847    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c738af7c8a2bf8c268c85ce4ee63415f3283e30607866516aa5cce48f91b1a5"} err="failed to get container status \"4c738af7c8a2bf8c268c85ce4ee63415f3283e30607866516aa5cce48f91b1a5\": rpc error: code = NotFound desc = could not find container \"4c738af7c8a2bf8c268c85ce4ee63415f3283e30607866516aa5cce48f91b1a5\": container with ID starting with 4c738af7c8a2bf8c268c85ce4ee63415f3283e30607866516aa5cce48f91b1a5 not found: ID does not exist"
Jul 29 10:59:39.986887 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.986862    2564 scope.go:117] "RemoveContainer" containerID="bf8c64780949e0300b214c8026ef8a90a4dd7e41319ce4f17ba0be1d14cae06a"
Jul 29 10:59:39.987076 ip-10-0-133-233 kubenswrapper[2564]: E0729 10:59:39.987058    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf8c64780949e0300b214c8026ef8a90a4dd7e41319ce4f17ba0be1d14cae06a\": container with ID starting with bf8c64780949e0300b214c8026ef8a90a4dd7e41319ce4f17ba0be1d14cae06a not found: ID does not exist" containerID="bf8c64780949e0300b214c8026ef8a90a4dd7e41319ce4f17ba0be1d14cae06a"
Jul 29 10:59:39.987119 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.987083    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf8c64780949e0300b214c8026ef8a90a4dd7e41319ce4f17ba0be1d14cae06a"} err="failed to get container status \"bf8c64780949e0300b214c8026ef8a90a4dd7e41319ce4f17ba0be1d14cae06a\": rpc error: code = NotFound desc = could not find container \"bf8c64780949e0300b214c8026ef8a90a4dd7e41319ce4f17ba0be1d14cae06a\": container with ID starting with bf8c64780949e0300b214c8026ef8a90a4dd7e41319ce4f17ba0be1d14cae06a not found: ID does not exist"
Jul 29 10:59:39.989863 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:39.989843    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-v2-predictor-865fc6b775-gs95d"]
Jul 29 10:59:41.342313 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:41.342286    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" path="/var/lib/kubelet/pods/4db18cb9-fe2f-4241-be77-49364295ac96/volumes"
Jul 29 10:59:45.485720 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.485684    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"]
Jul 29 10:59:45.486270 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.485987    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="storage-initializer"
Jul 29 10:59:45.486270 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.486002    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="storage-initializer"
Jul 29 10:59:45.486270 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.486018    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kserve-container"
Jul 29 10:59:45.486270 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.486024    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kserve-container"
Jul 29 10:59:45.486270 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.486030    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kube-rbac-proxy"
Jul 29 10:59:45.486270 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.486036    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kube-rbac-proxy"
Jul 29 10:59:45.486270 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.486104    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kube-rbac-proxy"
Jul 29 10:59:45.486270 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.486113    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4db18cb9-fe2f-4241-be77-49364295ac96" containerName="kserve-container"
Jul 29 10:59:45.490325 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.490304    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.493008 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.492989    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-primary-a23c96-predictor-serving-cert\""
Jul 29 10:59:45.493111 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.493072    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-primary-a23c96-kube-rbac-proxy-sar-config\""
Jul 29 10:59:45.494167 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.494143    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 29 10:59:45.494413 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.494396    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-6nnxs\""
Jul 29 10:59:45.494536 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.494459    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 29 10:59:45.494642 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.494613    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 29 10:59:45.494741 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.494723    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 29 10:59:45.498596 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.498576    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"]
Jul 29 10:59:45.570813 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.570767    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kube-rbac-proxy-sar-config\") pod \"isvc-primary-a23c96-predictor-5585c459fd-4c44k\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") " pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.571001 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.570835    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kserve-provision-location\") pod \"isvc-primary-a23c96-predictor-5585c459fd-4c44k\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") " pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.571001 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.570859    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-cabundle-cert\") pod \"isvc-primary-a23c96-predictor-5585c459fd-4c44k\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") " pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.571001 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.570884    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gqvb\" (UniqueName: \"kubernetes.io/projected/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kube-api-access-8gqvb\") pod \"isvc-primary-a23c96-predictor-5585c459fd-4c44k\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") " pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.571001 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.570988    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-proxy-tls\") pod \"isvc-primary-a23c96-predictor-5585c459fd-4c44k\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") " pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.671650 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.671620    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-proxy-tls\") pod \"isvc-primary-a23c96-predictor-5585c459fd-4c44k\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") " pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.671883 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.671679    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kube-rbac-proxy-sar-config\") pod \"isvc-primary-a23c96-predictor-5585c459fd-4c44k\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") " pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.671883 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.671707    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kserve-provision-location\") pod \"isvc-primary-a23c96-predictor-5585c459fd-4c44k\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") " pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.671883 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.671727    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-cabundle-cert\") pod \"isvc-primary-a23c96-predictor-5585c459fd-4c44k\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") " pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.671883 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.671788    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gqvb\" (UniqueName: \"kubernetes.io/projected/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kube-api-access-8gqvb\") pod \"isvc-primary-a23c96-predictor-5585c459fd-4c44k\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") " pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.672287 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.672263    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kserve-provision-location\") pod \"isvc-primary-a23c96-predictor-5585c459fd-4c44k\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") " pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.672462 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.672442    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kube-rbac-proxy-sar-config\") pod \"isvc-primary-a23c96-predictor-5585c459fd-4c44k\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") " pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.672524 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.672466    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-cabundle-cert\") pod \"isvc-primary-a23c96-predictor-5585c459fd-4c44k\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") " pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.674207 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.674183    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-proxy-tls\") pod \"isvc-primary-a23c96-predictor-5585c459fd-4c44k\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") " pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.681461 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.681442    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gqvb\" (UniqueName: \"kubernetes.io/projected/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kube-api-access-8gqvb\") pod \"isvc-primary-a23c96-predictor-5585c459fd-4c44k\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") " pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.801787 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.801708    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:45.931003 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.930973    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"]
Jul 29 10:59:45.933821 ip-10-0-133-233 kubenswrapper[2564]: W0729 10:59:45.933767    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e1ba43e_8fc8_4e12_bf6c_49101c48c616.slice/crio-7e4e68ea142aeba2d708e06a9205025d9d3be3dfbdfddb498b0f9372827a456a WatchSource:0}: Error finding container 7e4e68ea142aeba2d708e06a9205025d9d3be3dfbdfddb498b0f9372827a456a: Status 404 returned error can't find the container with id 7e4e68ea142aeba2d708e06a9205025d9d3be3dfbdfddb498b0f9372827a456a
Jul 29 10:59:45.985780 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:45.985751    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" event={"ID":"4e1ba43e-8fc8-4e12-bf6c-49101c48c616","Type":"ContainerStarted","Data":"7e4e68ea142aeba2d708e06a9205025d9d3be3dfbdfddb498b0f9372827a456a"}
Jul 29 10:59:46.991099 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:46.991059    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" event={"ID":"4e1ba43e-8fc8-4e12-bf6c-49101c48c616","Type":"ContainerStarted","Data":"62d12ba4f5db4829364b76c61758c590e6507d03d2eda560e56b11ce184bde5e"}
Jul 29 10:59:50.008047 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:50.008013    2564 generic.go:356] "Generic (PLEG): container finished" podID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerID="62d12ba4f5db4829364b76c61758c590e6507d03d2eda560e56b11ce184bde5e" exitCode=0
Jul 29 10:59:50.008441 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:50.008082    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" event={"ID":"4e1ba43e-8fc8-4e12-bf6c-49101c48c616","Type":"ContainerDied","Data":"62d12ba4f5db4829364b76c61758c590e6507d03d2eda560e56b11ce184bde5e"}
Jul 29 10:59:51.014253 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:51.014220    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" event={"ID":"4e1ba43e-8fc8-4e12-bf6c-49101c48c616","Type":"ContainerStarted","Data":"c5c43b516d94742c394fe683930c6b0546b3d6c0ffa6d1604952a6d77e65586f"}
Jul 29 10:59:51.014665 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:51.014278    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" event={"ID":"4e1ba43e-8fc8-4e12-bf6c-49101c48c616","Type":"ContainerStarted","Data":"0cead062e385044f660910f6ec7df3845e5bb18e1ce876c70b925408d64a1192"}
Jul 29 10:59:51.014665 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:51.014556    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:51.014780 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:51.014721    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:51.016178 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:51.016150    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 29 10:59:51.039563 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:51.039512    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" podStartSLOduration=2.315752426 podStartE2EDuration="6.039499272s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.723746846s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:59:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:59:51.038186428 +0000 UTC m=+1862.221542863" watchObservedRunningTime="2026-07-29 10:59:51.039499272 +0000 UTC m=+1862.222855687"
Jul 29 10:59:52.017765 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:52.017725    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 29 10:59:53.023124 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:53.023082    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 29 10:59:58.027293 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:58.027262    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 10:59:58.027763 ip-10-0-133-233 kubenswrapper[2564]: I0729 10:59:58.027735    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 29 11:00:08.028047 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:00:08.028006    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 29 11:00:18.028022 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:00:18.027979    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 29 11:00:28.028321 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:00:28.028281    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 29 11:00:38.028754 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:00:38.028711    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 29 11:00:48.028524 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:00:48.028483    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 29 11:00:58.028418 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:00:58.028376    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 29 11:01:08.028939 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:08.028906    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 11:01:44.777727 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:44.777639    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"]
Jul 29 11:01:44.778393 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:44.777932    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kserve-container" containerID="cri-o://0cead062e385044f660910f6ec7df3845e5bb18e1ce876c70b925408d64a1192" gracePeriod=30
Jul 29 11:01:44.778393 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:44.778014    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kube-rbac-proxy" containerID="cri-o://c5c43b516d94742c394fe683930c6b0546b3d6c0ffa6d1604952a6d77e65586f" gracePeriod=30
Jul 29 11:01:45.432467 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:45.432431    2564 generic.go:356] "Generic (PLEG): container finished" podID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerID="c5c43b516d94742c394fe683930c6b0546b3d6c0ffa6d1604952a6d77e65586f" exitCode=2
Jul 29 11:01:45.432663 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:45.432480    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" event={"ID":"4e1ba43e-8fc8-4e12-bf6c-49101c48c616","Type":"ContainerDied","Data":"c5c43b516d94742c394fe683930c6b0546b3d6c0ffa6d1604952a6d77e65586f"}
Jul 29 11:01:48.023918 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:48.023868    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 29 11:01:48.028215 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:48.028184    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 29 11:01:49.419422 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.419399    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 11:01:49.448723 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.448687    2564 generic.go:356] "Generic (PLEG): container finished" podID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerID="0cead062e385044f660910f6ec7df3845e5bb18e1ce876c70b925408d64a1192" exitCode=0
Jul 29 11:01:49.448901 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.448735    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"
Jul 29 11:01:49.448901 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.448760    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" event={"ID":"4e1ba43e-8fc8-4e12-bf6c-49101c48c616","Type":"ContainerDied","Data":"0cead062e385044f660910f6ec7df3845e5bb18e1ce876c70b925408d64a1192"}
Jul 29 11:01:49.448901 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.448853    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k" event={"ID":"4e1ba43e-8fc8-4e12-bf6c-49101c48c616","Type":"ContainerDied","Data":"7e4e68ea142aeba2d708e06a9205025d9d3be3dfbdfddb498b0f9372827a456a"}
Jul 29 11:01:49.448901 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.448896    2564 scope.go:117] "RemoveContainer" containerID="c5c43b516d94742c394fe683930c6b0546b3d6c0ffa6d1604952a6d77e65586f"
Jul 29 11:01:49.456472 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.456456    2564 scope.go:117] "RemoveContainer" containerID="0cead062e385044f660910f6ec7df3845e5bb18e1ce876c70b925408d64a1192"
Jul 29 11:01:49.463637 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.463620    2564 scope.go:117] "RemoveContainer" containerID="62d12ba4f5db4829364b76c61758c590e6507d03d2eda560e56b11ce184bde5e"
Jul 29 11:01:49.470575 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.470558    2564 scope.go:117] "RemoveContainer" containerID="c5c43b516d94742c394fe683930c6b0546b3d6c0ffa6d1604952a6d77e65586f"
Jul 29 11:01:49.470862 ip-10-0-133-233 kubenswrapper[2564]: E0729 11:01:49.470838    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5c43b516d94742c394fe683930c6b0546b3d6c0ffa6d1604952a6d77e65586f\": container with ID starting with c5c43b516d94742c394fe683930c6b0546b3d6c0ffa6d1604952a6d77e65586f not found: ID does not exist" containerID="c5c43b516d94742c394fe683930c6b0546b3d6c0ffa6d1604952a6d77e65586f"
Jul 29 11:01:49.470924 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.470871    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5c43b516d94742c394fe683930c6b0546b3d6c0ffa6d1604952a6d77e65586f"} err="failed to get container status \"c5c43b516d94742c394fe683930c6b0546b3d6c0ffa6d1604952a6d77e65586f\": rpc error: code = NotFound desc = could not find container \"c5c43b516d94742c394fe683930c6b0546b3d6c0ffa6d1604952a6d77e65586f\": container with ID starting with c5c43b516d94742c394fe683930c6b0546b3d6c0ffa6d1604952a6d77e65586f not found: ID does not exist"
Jul 29 11:01:49.470924 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.470888    2564 scope.go:117] "RemoveContainer" containerID="0cead062e385044f660910f6ec7df3845e5bb18e1ce876c70b925408d64a1192"
Jul 29 11:01:49.471142 ip-10-0-133-233 kubenswrapper[2564]: E0729 11:01:49.471125    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cead062e385044f660910f6ec7df3845e5bb18e1ce876c70b925408d64a1192\": container with ID starting with 0cead062e385044f660910f6ec7df3845e5bb18e1ce876c70b925408d64a1192 not found: ID does not exist" containerID="0cead062e385044f660910f6ec7df3845e5bb18e1ce876c70b925408d64a1192"
Jul 29 11:01:49.471197 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.471148    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cead062e385044f660910f6ec7df3845e5bb18e1ce876c70b925408d64a1192"} err="failed to get container status \"0cead062e385044f660910f6ec7df3845e5bb18e1ce876c70b925408d64a1192\": rpc error: code = NotFound desc = could not find container \"0cead062e385044f660910f6ec7df3845e5bb18e1ce876c70b925408d64a1192\": container with ID starting with 0cead062e385044f660910f6ec7df3845e5bb18e1ce876c70b925408d64a1192 not found: ID does not exist"
Jul 29 11:01:49.471197 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.471172    2564 scope.go:117] "RemoveContainer" containerID="62d12ba4f5db4829364b76c61758c590e6507d03d2eda560e56b11ce184bde5e"
Jul 29 11:01:49.471404 ip-10-0-133-233 kubenswrapper[2564]: E0729 11:01:49.471387    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62d12ba4f5db4829364b76c61758c590e6507d03d2eda560e56b11ce184bde5e\": container with ID starting with 62d12ba4f5db4829364b76c61758c590e6507d03d2eda560e56b11ce184bde5e not found: ID does not exist" containerID="62d12ba4f5db4829364b76c61758c590e6507d03d2eda560e56b11ce184bde5e"
Jul 29 11:01:49.471497 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.471410    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d12ba4f5db4829364b76c61758c590e6507d03d2eda560e56b11ce184bde5e"} err="failed to get container status \"62d12ba4f5db4829364b76c61758c590e6507d03d2eda560e56b11ce184bde5e\": rpc error: code = NotFound desc = could not find container \"62d12ba4f5db4829364b76c61758c590e6507d03d2eda560e56b11ce184bde5e\": container with ID starting with 62d12ba4f5db4829364b76c61758c590e6507d03d2eda560e56b11ce184bde5e not found: ID does not exist"
Jul 29 11:01:49.491706 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.491687    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gqvb\" (UniqueName: \"kubernetes.io/projected/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kube-api-access-8gqvb\") pod \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") "
Jul 29 11:01:49.491826 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.491719    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-cabundle-cert\") pod \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") "
Jul 29 11:01:49.491826 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.491747    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kserve-provision-location\") pod \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") "
Jul 29 11:01:49.491826 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.491764    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-proxy-tls\") pod \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") "
Jul 29 11:01:49.491826 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.491783    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kube-rbac-proxy-sar-config\") pod \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\" (UID: \"4e1ba43e-8fc8-4e12-bf6c-49101c48c616\") "
Jul 29 11:01:49.492099 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.492075    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4e1ba43e-8fc8-4e12-bf6c-49101c48c616" (UID: "4e1ba43e-8fc8-4e12-bf6c-49101c48c616"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 11:01:49.492143 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.492101    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "4e1ba43e-8fc8-4e12-bf6c-49101c48c616" (UID: "4e1ba43e-8fc8-4e12-bf6c-49101c48c616"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 11:01:49.492179 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.492154    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4e1ba43e-8fc8-4e12-bf6c-49101c48c616" (UID: "4e1ba43e-8fc8-4e12-bf6c-49101c48c616"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 11:01:49.493781 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.493748    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4e1ba43e-8fc8-4e12-bf6c-49101c48c616" (UID: "4e1ba43e-8fc8-4e12-bf6c-49101c48c616"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 11:01:49.493858 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.493778    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kube-api-access-8gqvb" (OuterVolumeSpecName: "kube-api-access-8gqvb") pod "4e1ba43e-8fc8-4e12-bf6c-49101c48c616" (UID: "4e1ba43e-8fc8-4e12-bf6c-49101c48c616"). InnerVolumeSpecName "kube-api-access-8gqvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 11:01:49.593274 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.593237    2564 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-cabundle-cert\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 11:01:49.593274 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.593266    2564 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kserve-provision-location\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 11:01:49.593274 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.593280    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-proxy-tls\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 11:01:49.593490 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.593291    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 11:01:49.593490 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.593304    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gqvb\" (UniqueName: \"kubernetes.io/projected/4e1ba43e-8fc8-4e12-bf6c-49101c48c616-kube-api-access-8gqvb\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 11:01:49.770997 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.770972    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"]
Jul 29 11:01:49.774507 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:49.774479    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-a23c96-predictor-5585c459fd-4c44k"]
Jul 29 11:01:51.342813 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:01:51.342769    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" path="/var/lib/kubelet/pods/4e1ba43e-8fc8-4e12-bf6c-49101c48c616/volumes"
Jul 29 11:02:05.130296 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.130259    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"]
Jul 29 11:02:05.130670 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.130536    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kserve-container"
Jul 29 11:02:05.130670 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.130547    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kserve-container"
Jul 29 11:02:05.130670 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.130562    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kube-rbac-proxy"
Jul 29 11:02:05.130670 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.130567    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kube-rbac-proxy"
Jul 29 11:02:05.130670 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.130577    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="storage-initializer"
Jul 29 11:02:05.130670 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.130583    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="storage-initializer"
Jul 29 11:02:05.130670 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.130631    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kube-rbac-proxy"
Jul 29 11:02:05.130670 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.130639    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e1ba43e-8fc8-4e12-bf6c-49101c48c616" containerName="kserve-container"
Jul 29 11:02:05.133678 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.133657    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:05.136388 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.136359    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 29 11:02:05.136534 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.136407    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"raw-sklearn-0bfc7-predictor-serving-cert\""
Jul 29 11:02:05.136534 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.136468    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 29 11:02:05.136740 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.136722    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 29 11:02:05.136863 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.136837    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-6nnxs\""
Jul 29 11:02:05.136964 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.136857    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"raw-sklearn-0bfc7-kube-rbac-proxy-sar-config\""
Jul 29 11:02:05.137450 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.137435    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 29 11:02:05.143397 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.143376    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"]
Jul 29 11:02:05.211971 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.211933    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fb768d22-270f-4a99-ae5f-a5af48203428-cabundle-cert\") pod \"raw-sklearn-0bfc7-predictor-747789688c-89kk6\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") " pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:05.212145 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.211980    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fb768d22-270f-4a99-ae5f-a5af48203428-proxy-tls\") pod \"raw-sklearn-0bfc7-predictor-747789688c-89kk6\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") " pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:05.212145 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.212008    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8skv\" (UniqueName: \"kubernetes.io/projected/fb768d22-270f-4a99-ae5f-a5af48203428-kube-api-access-g8skv\") pod \"raw-sklearn-0bfc7-predictor-747789688c-89kk6\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") " pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:05.212145 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.212050    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fb768d22-270f-4a99-ae5f-a5af48203428-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-0bfc7-predictor-747789688c-89kk6\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") " pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:05.212145 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.212083    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fb768d22-270f-4a99-ae5f-a5af48203428-kserve-provision-location\") pod \"raw-sklearn-0bfc7-predictor-747789688c-89kk6\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") " pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:05.313236 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.313198    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fb768d22-270f-4a99-ae5f-a5af48203428-cabundle-cert\") pod \"raw-sklearn-0bfc7-predictor-747789688c-89kk6\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") " pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:05.313236 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.313244    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fb768d22-270f-4a99-ae5f-a5af48203428-proxy-tls\") pod \"raw-sklearn-0bfc7-predictor-747789688c-89kk6\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") " pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:05.313447 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.313269    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8skv\" (UniqueName: \"kubernetes.io/projected/fb768d22-270f-4a99-ae5f-a5af48203428-kube-api-access-g8skv\") pod \"raw-sklearn-0bfc7-predictor-747789688c-89kk6\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") " pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:05.313447 ip-10-0-133-233 kubenswrapper[2564]: E0729 11:02:05.313364    2564 secret.go:189] Couldn't get secret kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-serving-cert: secret "raw-sklearn-0bfc7-predictor-serving-cert" not found
Jul 29 11:02:05.313447 ip-10-0-133-233 kubenswrapper[2564]: E0729 11:02:05.313429    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb768d22-270f-4a99-ae5f-a5af48203428-proxy-tls podName:fb768d22-270f-4a99-ae5f-a5af48203428 nodeName:}" failed. No retries permitted until 2026-07-29 11:02:05.813408417 +0000 UTC m=+1996.996764810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/fb768d22-270f-4a99-ae5f-a5af48203428-proxy-tls") pod "raw-sklearn-0bfc7-predictor-747789688c-89kk6" (UID: "fb768d22-270f-4a99-ae5f-a5af48203428") : secret "raw-sklearn-0bfc7-predictor-serving-cert" not found
Jul 29 11:02:05.313594 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.313370    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fb768d22-270f-4a99-ae5f-a5af48203428-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-0bfc7-predictor-747789688c-89kk6\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") " pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:05.313594 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.313528    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fb768d22-270f-4a99-ae5f-a5af48203428-kserve-provision-location\") pod \"raw-sklearn-0bfc7-predictor-747789688c-89kk6\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") " pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:05.313887 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.313868    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fb768d22-270f-4a99-ae5f-a5af48203428-kserve-provision-location\") pod \"raw-sklearn-0bfc7-predictor-747789688c-89kk6\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") " pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:05.314028 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.314010    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fb768d22-270f-4a99-ae5f-a5af48203428-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-0bfc7-predictor-747789688c-89kk6\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") " pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:05.314069 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.314038    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fb768d22-270f-4a99-ae5f-a5af48203428-cabundle-cert\") pod \"raw-sklearn-0bfc7-predictor-747789688c-89kk6\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") " pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:05.321950 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.321931    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8skv\" (UniqueName: \"kubernetes.io/projected/fb768d22-270f-4a99-ae5f-a5af48203428-kube-api-access-g8skv\") pod \"raw-sklearn-0bfc7-predictor-747789688c-89kk6\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") " pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:05.816886 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.816847    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fb768d22-270f-4a99-ae5f-a5af48203428-proxy-tls\") pod \"raw-sklearn-0bfc7-predictor-747789688c-89kk6\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") " pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:05.819204 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:05.819175    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fb768d22-270f-4a99-ae5f-a5af48203428-proxy-tls\") pod \"raw-sklearn-0bfc7-predictor-747789688c-89kk6\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") " pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:06.045050 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:06.045012    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:06.172633 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:06.172606    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"]
Jul 29 11:02:06.175021 ip-10-0-133-233 kubenswrapper[2564]: W0729 11:02:06.174995    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb768d22_270f_4a99_ae5f_a5af48203428.slice/crio-a1945e8dd2b97a2a9af1eb0e947d7b401b23cb3a9fe437c37cf5e01ba5b647c1 WatchSource:0}: Error finding container a1945e8dd2b97a2a9af1eb0e947d7b401b23cb3a9fe437c37cf5e01ba5b647c1: Status 404 returned error can't find the container with id a1945e8dd2b97a2a9af1eb0e947d7b401b23cb3a9fe437c37cf5e01ba5b647c1
Jul 29 11:02:06.515760 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:06.515729    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" event={"ID":"fb768d22-270f-4a99-ae5f-a5af48203428","Type":"ContainerStarted","Data":"20a4c0b804ffc01e23a7f86c466d02e6b81a2bbad3b7704b3200098ce767a7c6"}
Jul 29 11:02:06.515965 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:06.515816    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" event={"ID":"fb768d22-270f-4a99-ae5f-a5af48203428","Type":"ContainerStarted","Data":"a1945e8dd2b97a2a9af1eb0e947d7b401b23cb3a9fe437c37cf5e01ba5b647c1"}
Jul 29 11:02:10.531042 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:10.531006    2564 generic.go:356] "Generic (PLEG): container finished" podID="fb768d22-270f-4a99-ae5f-a5af48203428" containerID="20a4c0b804ffc01e23a7f86c466d02e6b81a2bbad3b7704b3200098ce767a7c6" exitCode=0
Jul 29 11:02:10.531474 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:10.531079    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" event={"ID":"fb768d22-270f-4a99-ae5f-a5af48203428","Type":"ContainerDied","Data":"20a4c0b804ffc01e23a7f86c466d02e6b81a2bbad3b7704b3200098ce767a7c6"}
Jul 29 11:02:11.537096 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:11.537063    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" event={"ID":"fb768d22-270f-4a99-ae5f-a5af48203428","Type":"ContainerStarted","Data":"ee4c48ea094d01a72f57ec4823b8e97fcb74ac5cc59788dccb43acf0141f3dc2"}
Jul 29 11:02:11.537536 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:11.537134    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" event={"ID":"fb768d22-270f-4a99-ae5f-a5af48203428","Type":"ContainerStarted","Data":"109c5067fbf3cd84c2029f0680fcfabec28843ab437a9638b2db5bbc364485be"}
Jul 29 11:02:11.537536 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:11.537358    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:11.557103 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:11.557053    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" podStartSLOduration=2.7425042619999997 podStartE2EDuration="6.557001737s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.814497475s" isStatefulPod=false podCreationTimestamp="2026-07-29 11:02:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 11:02:11.555835439 +0000 UTC m=+2002.739191863" watchObservedRunningTime="2026-07-29 11:02:11.557001737 +0000 UTC m=+2002.740358152"
Jul 29 11:02:12.541923 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:12.541888    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:12.543346 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:12.543316    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Jul 29 11:02:13.545369 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:13.545331    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Jul 29 11:02:18.549737 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:18.549701    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:02:18.550309 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:18.550283    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Jul 29 11:02:28.550501 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:28.550459    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Jul 29 11:02:38.550963 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:38.550925    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Jul 29 11:02:48.550388 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:48.550343    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Jul 29 11:02:58.551240 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:02:58.551191    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Jul 29 11:03:08.550989 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:08.550950    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Jul 29 11:03:18.550725 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:18.550682    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Jul 29 11:03:28.550968 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:28.550940    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:03:35.276104 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:35.276064    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"]
Jul 29 11:03:35.276603 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:35.276365    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kserve-container" containerID="cri-o://109c5067fbf3cd84c2029f0680fcfabec28843ab437a9638b2db5bbc364485be" gracePeriod=30
Jul 29 11:03:35.276603 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:35.276396    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kube-rbac-proxy" containerID="cri-o://ee4c48ea094d01a72f57ec4823b8e97fcb74ac5cc59788dccb43acf0141f3dc2" gracePeriod=30
Jul 29 11:03:35.843159 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:35.843121    2564 generic.go:356] "Generic (PLEG): container finished" podID="fb768d22-270f-4a99-ae5f-a5af48203428" containerID="ee4c48ea094d01a72f57ec4823b8e97fcb74ac5cc59788dccb43acf0141f3dc2" exitCode=2
Jul 29 11:03:35.843159 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:35.843159    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" event={"ID":"fb768d22-270f-4a99-ae5f-a5af48203428","Type":"ContainerDied","Data":"ee4c48ea094d01a72f57ec4823b8e97fcb74ac5cc59788dccb43acf0141f3dc2"}
Jul 29 11:03:38.546278 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:38.546236    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Jul 29 11:03:38.550594 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:38.550570    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Jul 29 11:03:40.020953 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.020923    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:03:40.033615 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.033590    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fb768d22-270f-4a99-ae5f-a5af48203428-cabundle-cert\") pod \"fb768d22-270f-4a99-ae5f-a5af48203428\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") "
Jul 29 11:03:40.033739 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.033621    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8skv\" (UniqueName: \"kubernetes.io/projected/fb768d22-270f-4a99-ae5f-a5af48203428-kube-api-access-g8skv\") pod \"fb768d22-270f-4a99-ae5f-a5af48203428\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") "
Jul 29 11:03:40.033739 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.033665    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fb768d22-270f-4a99-ae5f-a5af48203428-kube-rbac-proxy-sar-config\") pod \"fb768d22-270f-4a99-ae5f-a5af48203428\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") "
Jul 29 11:03:40.034184 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.033754    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fb768d22-270f-4a99-ae5f-a5af48203428-kserve-provision-location\") pod \"fb768d22-270f-4a99-ae5f-a5af48203428\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") "
Jul 29 11:03:40.034184 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.033830    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fb768d22-270f-4a99-ae5f-a5af48203428-proxy-tls\") pod \"fb768d22-270f-4a99-ae5f-a5af48203428\" (UID: \"fb768d22-270f-4a99-ae5f-a5af48203428\") "
Jul 29 11:03:40.034184 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.034058    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb768d22-270f-4a99-ae5f-a5af48203428-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "fb768d22-270f-4a99-ae5f-a5af48203428" (UID: "fb768d22-270f-4a99-ae5f-a5af48203428"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 11:03:40.034184 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.034071    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb768d22-270f-4a99-ae5f-a5af48203428-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "fb768d22-270f-4a99-ae5f-a5af48203428" (UID: "fb768d22-270f-4a99-ae5f-a5af48203428"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 11:03:40.034184 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.034065    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb768d22-270f-4a99-ae5f-a5af48203428-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "fb768d22-270f-4a99-ae5f-a5af48203428" (UID: "fb768d22-270f-4a99-ae5f-a5af48203428"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 11:03:40.035944 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.035921    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb768d22-270f-4a99-ae5f-a5af48203428-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "fb768d22-270f-4a99-ae5f-a5af48203428" (UID: "fb768d22-270f-4a99-ae5f-a5af48203428"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 11:03:40.035944 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.035929    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb768d22-270f-4a99-ae5f-a5af48203428-kube-api-access-g8skv" (OuterVolumeSpecName: "kube-api-access-g8skv") pod "fb768d22-270f-4a99-ae5f-a5af48203428" (UID: "fb768d22-270f-4a99-ae5f-a5af48203428"). InnerVolumeSpecName "kube-api-access-g8skv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 11:03:40.134769 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.134685    2564 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fb768d22-270f-4a99-ae5f-a5af48203428-cabundle-cert\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 11:03:40.134769 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.134710    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8skv\" (UniqueName: \"kubernetes.io/projected/fb768d22-270f-4a99-ae5f-a5af48203428-kube-api-access-g8skv\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 11:03:40.134769 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.134719    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fb768d22-270f-4a99-ae5f-a5af48203428-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 11:03:40.134769 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.134729    2564 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fb768d22-270f-4a99-ae5f-a5af48203428-kserve-provision-location\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 11:03:40.134769 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.134739    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fb768d22-270f-4a99-ae5f-a5af48203428-proxy-tls\") on node \"ip-10-0-133-233.ec2.internal\" DevicePath \"\""
Jul 29 11:03:40.864390 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.864356    2564 generic.go:356] "Generic (PLEG): container finished" podID="fb768d22-270f-4a99-ae5f-a5af48203428" containerID="109c5067fbf3cd84c2029f0680fcfabec28843ab437a9638b2db5bbc364485be" exitCode=0
Jul 29 11:03:40.864571 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.864403    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" event={"ID":"fb768d22-270f-4a99-ae5f-a5af48203428","Type":"ContainerDied","Data":"109c5067fbf3cd84c2029f0680fcfabec28843ab437a9638b2db5bbc364485be"}
Jul 29 11:03:40.864571 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.864414    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"
Jul 29 11:03:40.864571 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.864452    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6" event={"ID":"fb768d22-270f-4a99-ae5f-a5af48203428","Type":"ContainerDied","Data":"a1945e8dd2b97a2a9af1eb0e947d7b401b23cb3a9fe437c37cf5e01ba5b647c1"}
Jul 29 11:03:40.864571 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.864481    2564 scope.go:117] "RemoveContainer" containerID="ee4c48ea094d01a72f57ec4823b8e97fcb74ac5cc59788dccb43acf0141f3dc2"
Jul 29 11:03:40.873119 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.873100    2564 scope.go:117] "RemoveContainer" containerID="109c5067fbf3cd84c2029f0680fcfabec28843ab437a9638b2db5bbc364485be"
Jul 29 11:03:40.880130 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.880111    2564 scope.go:117] "RemoveContainer" containerID="20a4c0b804ffc01e23a7f86c466d02e6b81a2bbad3b7704b3200098ce767a7c6"
Jul 29 11:03:40.886544 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.886517    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"]
Jul 29 11:03:40.887242 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.887228    2564 scope.go:117] "RemoveContainer" containerID="ee4c48ea094d01a72f57ec4823b8e97fcb74ac5cc59788dccb43acf0141f3dc2"
Jul 29 11:03:40.887532 ip-10-0-133-233 kubenswrapper[2564]: E0729 11:03:40.887509    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee4c48ea094d01a72f57ec4823b8e97fcb74ac5cc59788dccb43acf0141f3dc2\": container with ID starting with ee4c48ea094d01a72f57ec4823b8e97fcb74ac5cc59788dccb43acf0141f3dc2 not found: ID does not exist" containerID="ee4c48ea094d01a72f57ec4823b8e97fcb74ac5cc59788dccb43acf0141f3dc2"
Jul 29 11:03:40.887575 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.887545    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee4c48ea094d01a72f57ec4823b8e97fcb74ac5cc59788dccb43acf0141f3dc2"} err="failed to get container status \"ee4c48ea094d01a72f57ec4823b8e97fcb74ac5cc59788dccb43acf0141f3dc2\": rpc error: code = NotFound desc = could not find container \"ee4c48ea094d01a72f57ec4823b8e97fcb74ac5cc59788dccb43acf0141f3dc2\": container with ID starting with ee4c48ea094d01a72f57ec4823b8e97fcb74ac5cc59788dccb43acf0141f3dc2 not found: ID does not exist"
Jul 29 11:03:40.887575 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.887570    2564 scope.go:117] "RemoveContainer" containerID="109c5067fbf3cd84c2029f0680fcfabec28843ab437a9638b2db5bbc364485be"
Jul 29 11:03:40.887854 ip-10-0-133-233 kubenswrapper[2564]: E0729 11:03:40.887831    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"109c5067fbf3cd84c2029f0680fcfabec28843ab437a9638b2db5bbc364485be\": container with ID starting with 109c5067fbf3cd84c2029f0680fcfabec28843ab437a9638b2db5bbc364485be not found: ID does not exist" containerID="109c5067fbf3cd84c2029f0680fcfabec28843ab437a9638b2db5bbc364485be"
Jul 29 11:03:40.887926 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.887857    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"109c5067fbf3cd84c2029f0680fcfabec28843ab437a9638b2db5bbc364485be"} err="failed to get container status \"109c5067fbf3cd84c2029f0680fcfabec28843ab437a9638b2db5bbc364485be\": rpc error: code = NotFound desc = could not find container \"109c5067fbf3cd84c2029f0680fcfabec28843ab437a9638b2db5bbc364485be\": container with ID starting with 109c5067fbf3cd84c2029f0680fcfabec28843ab437a9638b2db5bbc364485be not found: ID does not exist"
Jul 29 11:03:40.887926 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.887874    2564 scope.go:117] "RemoveContainer" containerID="20a4c0b804ffc01e23a7f86c466d02e6b81a2bbad3b7704b3200098ce767a7c6"
Jul 29 11:03:40.888259 ip-10-0-133-233 kubenswrapper[2564]: E0729 11:03:40.888200    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20a4c0b804ffc01e23a7f86c466d02e6b81a2bbad3b7704b3200098ce767a7c6\": container with ID starting with 20a4c0b804ffc01e23a7f86c466d02e6b81a2bbad3b7704b3200098ce767a7c6 not found: ID does not exist" containerID="20a4c0b804ffc01e23a7f86c466d02e6b81a2bbad3b7704b3200098ce767a7c6"
Jul 29 11:03:40.888259 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.888229    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20a4c0b804ffc01e23a7f86c466d02e6b81a2bbad3b7704b3200098ce767a7c6"} err="failed to get container status \"20a4c0b804ffc01e23a7f86c466d02e6b81a2bbad3b7704b3200098ce767a7c6\": rpc error: code = NotFound desc = could not find container \"20a4c0b804ffc01e23a7f86c466d02e6b81a2bbad3b7704b3200098ce767a7c6\": container with ID starting with 20a4c0b804ffc01e23a7f86c466d02e6b81a2bbad3b7704b3200098ce767a7c6 not found: ID does not exist"
Jul 29 11:03:40.889896 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:40.889876    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-0bfc7-predictor-747789688c-89kk6"]
Jul 29 11:03:41.342518 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:03:41.342488    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" path="/var/lib/kubelet/pods/fb768d22-270f-4a99-ae5f-a5af48203428/volumes"
Jul 29 11:05:49.559150 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:49.559122    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rvqcw_a6d54b5d-72a9-412c-8dbc-e4993832122a/global-pull-secret-syncer/0.log"
Jul 29 11:05:49.641713 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:49.641685    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dclkn_9a04cac1-c3f9-42db-9339-54caac308ad2/konnectivity-agent/0.log"
Jul 29 11:05:49.702957 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:49.702930    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-233.ec2.internal_ee950c7e37cd6646704499e83e8cb37d/haproxy/0.log"
Jul 29 11:05:54.453129 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:54.453097    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6ps2g_4decb700-a44b-48d9-9e6d-726e45aac336/node-exporter/0.log"
Jul 29 11:05:54.472306 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:54.472280    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6ps2g_4decb700-a44b-48d9-9e6d-726e45aac336/kube-rbac-proxy/0.log"
Jul 29 11:05:54.489861 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:54.489837    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6ps2g_4decb700-a44b-48d9-9e6d-726e45aac336/init-textfile/0.log"
Jul 29 11:05:57.826869 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:57.826843    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-fh4qq_e6ba5cdf-2ce5-4286-8a74-6f9f0b89e75d/volume-data-source-validator/0.log"
Jul 29 11:05:58.342974 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.342945    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"]
Jul 29 11:05:58.343210 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.343198    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kube-rbac-proxy"
Jul 29 11:05:58.343264 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.343211    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kube-rbac-proxy"
Jul 29 11:05:58.343264 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.343224    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kserve-container"
Jul 29 11:05:58.343264 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.343229    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kserve-container"
Jul 29 11:05:58.343264 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.343237    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="storage-initializer"
Jul 29 11:05:58.343264 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.343242    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="storage-initializer"
Jul 29 11:05:58.343417 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.343301    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kube-rbac-proxy"
Jul 29 11:05:58.343417 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.343309    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb768d22-270f-4a99-ae5f-a5af48203428" containerName="kserve-container"
Jul 29 11:05:58.346090 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.346068    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.348910 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.348888    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mxlpr\"/\"kube-root-ca.crt\""
Jul 29 11:05:58.350112 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.350094    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mxlpr\"/\"default-dockercfg-dc47v\""
Jul 29 11:05:58.350227 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.350120    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mxlpr\"/\"openshift-service-ca.crt\""
Jul 29 11:05:58.355723 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.355703    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"]
Jul 29 11:05:58.461983 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.461950    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp2qf\" (UniqueName: \"kubernetes.io/projected/f531aa71-403d-451f-ae58-d55140278acd-kube-api-access-tp2qf\") pod \"perf-node-gather-daemonset-tvk24\" (UID: \"f531aa71-403d-451f-ae58-d55140278acd\") " pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.462157 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.462008    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f531aa71-403d-451f-ae58-d55140278acd-lib-modules\") pod \"perf-node-gather-daemonset-tvk24\" (UID: \"f531aa71-403d-451f-ae58-d55140278acd\") " pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.462157 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.462036    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f531aa71-403d-451f-ae58-d55140278acd-proc\") pod \"perf-node-gather-daemonset-tvk24\" (UID: \"f531aa71-403d-451f-ae58-d55140278acd\") " pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.462157 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.462067    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f531aa71-403d-451f-ae58-d55140278acd-podres\") pod \"perf-node-gather-daemonset-tvk24\" (UID: \"f531aa71-403d-451f-ae58-d55140278acd\") " pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.462157 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.462108    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f531aa71-403d-451f-ae58-d55140278acd-sys\") pod \"perf-node-gather-daemonset-tvk24\" (UID: \"f531aa71-403d-451f-ae58-d55140278acd\") " pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.563421 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.563374    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f531aa71-403d-451f-ae58-d55140278acd-sys\") pod \"perf-node-gather-daemonset-tvk24\" (UID: \"f531aa71-403d-451f-ae58-d55140278acd\") " pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.563421 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.563430    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp2qf\" (UniqueName: \"kubernetes.io/projected/f531aa71-403d-451f-ae58-d55140278acd-kube-api-access-tp2qf\") pod \"perf-node-gather-daemonset-tvk24\" (UID: \"f531aa71-403d-451f-ae58-d55140278acd\") " pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.563618 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.563459    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f531aa71-403d-451f-ae58-d55140278acd-lib-modules\") pod \"perf-node-gather-daemonset-tvk24\" (UID: \"f531aa71-403d-451f-ae58-d55140278acd\") " pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.563618 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.563499    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f531aa71-403d-451f-ae58-d55140278acd-proc\") pod \"perf-node-gather-daemonset-tvk24\" (UID: \"f531aa71-403d-451f-ae58-d55140278acd\") " pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.563618 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.563510    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f531aa71-403d-451f-ae58-d55140278acd-sys\") pod \"perf-node-gather-daemonset-tvk24\" (UID: \"f531aa71-403d-451f-ae58-d55140278acd\") " pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.563618 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.563554    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f531aa71-403d-451f-ae58-d55140278acd-podres\") pod \"perf-node-gather-daemonset-tvk24\" (UID: \"f531aa71-403d-451f-ae58-d55140278acd\") " pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.563618 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.563561    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f531aa71-403d-451f-ae58-d55140278acd-proc\") pod \"perf-node-gather-daemonset-tvk24\" (UID: \"f531aa71-403d-451f-ae58-d55140278acd\") " pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.563779 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.563638    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f531aa71-403d-451f-ae58-d55140278acd-podres\") pod \"perf-node-gather-daemonset-tvk24\" (UID: \"f531aa71-403d-451f-ae58-d55140278acd\") " pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.563779 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.563657    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f531aa71-403d-451f-ae58-d55140278acd-lib-modules\") pod \"perf-node-gather-daemonset-tvk24\" (UID: \"f531aa71-403d-451f-ae58-d55140278acd\") " pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.571976 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.571955    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp2qf\" (UniqueName: \"kubernetes.io/projected/f531aa71-403d-451f-ae58-d55140278acd-kube-api-access-tp2qf\") pod \"perf-node-gather-daemonset-tvk24\" (UID: \"f531aa71-403d-451f-ae58-d55140278acd\") " pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.655900 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.655833    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:58.756709 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.756671    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hhbsk_73102ac7-c5cf-43b4-a289-4976d3e902e9/dns/0.log"
Jul 29 11:05:58.778072 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.778046    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hhbsk_73102ac7-c5cf-43b4-a289-4976d3e902e9/kube-rbac-proxy/0.log"
Jul 29 11:05:58.778178 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.778123    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"]
Jul 29 11:05:58.780432 ip-10-0-133-233 kubenswrapper[2564]: W0729 11:05:58.780410    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf531aa71_403d_451f_ae58_d55140278acd.slice/crio-a52b47b08b37b16234f8662936c569d801afd5bb65fc1641c1025ce176755ef8 WatchSource:0}: Error finding container a52b47b08b37b16234f8662936c569d801afd5bb65fc1641c1025ce176755ef8: Status 404 returned error can't find the container with id a52b47b08b37b16234f8662936c569d801afd5bb65fc1641c1025ce176755ef8
Jul 29 11:05:58.884588 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:58.884560    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wgrdq_7d9c53a1-f5af-4bfc-81ba-4703f2fc039b/dns-node-resolver/0.log"
Jul 29 11:05:59.338641 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:59.338610    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j5xql_3ffc02ea-68ae-4d60-82a4-01d159cd668c/node-ca/0.log"
Jul 29 11:05:59.351443 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:59.351419    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24" event={"ID":"f531aa71-403d-451f-ae58-d55140278acd","Type":"ContainerStarted","Data":"a387695e9fee9d1559c7c2dd85f9d40eaaaa343fe71ef6d89c1d8d73752c9b08"}
Jul 29 11:05:59.351582 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:59.351460    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24" event={"ID":"f531aa71-403d-451f-ae58-d55140278acd","Type":"ContainerStarted","Data":"a52b47b08b37b16234f8662936c569d801afd5bb65fc1641c1025ce176755ef8"}
Jul 29 11:05:59.351582 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:59.351577    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:05:59.368941 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:05:59.368895    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24" podStartSLOduration=1.368882183 podStartE2EDuration="1.368882183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 11:05:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 11:05:59.368471903 +0000 UTC m=+2230.551828317" watchObservedRunningTime="2026-07-29 11:05:59.368882183 +0000 UTC m=+2230.552238597"
Jul 29 11:06:00.324972 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:00.324944    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8dxdp_04cbd6e9-e56f-4e33-bc1a-901eeec3248d/serve-healthcheck-canary/0.log"
Jul 29 11:06:00.676841 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:00.676740    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-vm85k_4875bd25-9d39-432d-bffa-2f422607180d/insights-operator/0.log"
Jul 29 11:06:00.677770 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:00.677753    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-vm85k_4875bd25-9d39-432d-bffa-2f422607180d/insights-operator/1.log"
Jul 29 11:06:00.693972 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:00.693954    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-brmw9_9aad0867-9ee3-4e88-9704-a20cf319dbe8/kube-rbac-proxy/0.log"
Jul 29 11:06:00.710344 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:00.710323    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-brmw9_9aad0867-9ee3-4e88-9704-a20cf319dbe8/exporter/0.log"
Jul 29 11:06:00.728814 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:00.728784    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-brmw9_9aad0867-9ee3-4e88-9704-a20cf319dbe8/extractor/0.log"
Jul 29 11:06:02.985891 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:02.985859    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-65f566c74b-vtfxb_aeecb00a-3e07-473e-848f-24261afa6a4a/manager/0.log"
Jul 29 11:06:05.364661 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:05.364630    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mxlpr/perf-node-gather-daemonset-tvk24"
Jul 29 11:06:07.144863 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:07.144835    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5_22f26925-6582-4316-a1b1-a451fe0ecba8/kube-storage-version-migrator-operator/1.log"
Jul 29 11:06:07.145695 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:07.145677    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-nzkm5_22f26925-6582-4316-a1b1-a451fe0ecba8/kube-storage-version-migrator-operator/0.log"
Jul 29 11:06:08.462831 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:08.462732    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rjgvn_dbf16cf7-3ffb-4dd6-a559-0d47817db5cf/kube-multus-additional-cni-plugins/0.log"
Jul 29 11:06:08.480314 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:08.480291    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rjgvn_dbf16cf7-3ffb-4dd6-a559-0d47817db5cf/egress-router-binary-copy/0.log"
Jul 29 11:06:08.496980 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:08.496954    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rjgvn_dbf16cf7-3ffb-4dd6-a559-0d47817db5cf/cni-plugins/0.log"
Jul 29 11:06:08.513312 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:08.513283    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rjgvn_dbf16cf7-3ffb-4dd6-a559-0d47817db5cf/bond-cni-plugin/0.log"
Jul 29 11:06:08.530523 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:08.530500    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rjgvn_dbf16cf7-3ffb-4dd6-a559-0d47817db5cf/routeoverride-cni/0.log"
Jul 29 11:06:08.548449 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:08.548421    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rjgvn_dbf16cf7-3ffb-4dd6-a559-0d47817db5cf/whereabouts-cni-bincopy/0.log"
Jul 29 11:06:08.568816 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:08.568784    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rjgvn_dbf16cf7-3ffb-4dd6-a559-0d47817db5cf/whereabouts-cni/0.log"
Jul 29 11:06:08.594396 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:08.594369    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mtx8j_985d7ab1-89e7-4f96-908f-559b300397ea/kube-multus/0.log"
Jul 29 11:06:08.740090 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:08.740067    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-x5nh2_d598bd41-0d98-42fb-b082-8582a3b3b5ef/network-metrics-daemon/0.log"
Jul 29 11:06:08.757059 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:08.757038    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-x5nh2_d598bd41-0d98-42fb-b082-8582a3b3b5ef/kube-rbac-proxy/0.log"
Jul 29 11:06:09.756359 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:09.756334    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psdkf_1ab4b50a-79d2-43f1-b4c0-e7459adb20c7/ovn-controller/0.log"
Jul 29 11:06:09.773228 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:09.773201    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psdkf_1ab4b50a-79d2-43f1-b4c0-e7459adb20c7/ovn-acl-logging/0.log"
Jul 29 11:06:09.794987 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:09.794964    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psdkf_1ab4b50a-79d2-43f1-b4c0-e7459adb20c7/kube-rbac-proxy-node/0.log"
Jul 29 11:06:09.813580 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:09.813556    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psdkf_1ab4b50a-79d2-43f1-b4c0-e7459adb20c7/kube-rbac-proxy-ovn-metrics/0.log"
Jul 29 11:06:09.830370 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:09.830299    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psdkf_1ab4b50a-79d2-43f1-b4c0-e7459adb20c7/northd/0.log"
Jul 29 11:06:09.848734 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:09.848713    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psdkf_1ab4b50a-79d2-43f1-b4c0-e7459adb20c7/nbdb/0.log"
Jul 29 11:06:09.867554 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:09.867529    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psdkf_1ab4b50a-79d2-43f1-b4c0-e7459adb20c7/sbdb/0.log"
Jul 29 11:06:09.902788 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:09.902759    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-psdkf_1ab4b50a-79d2-43f1-b4c0-e7459adb20c7/ovnkube-controller/0.log"
Jul 29 11:06:11.001582 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:11.001553    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-fp5xd_33f96a45-1e06-4af4-bcbe-7e494cfcbfe2/network-check-target-container/0.log"
Jul 29 11:06:12.010176 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:12.010148    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-2mz7m_c8eca8a0-11a6-488c-8856-53c63d71c5bf/iptables-alerter/0.log"
Jul 29 11:06:12.663034 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:12.663005    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-9jjlf_0ad2e5c5-ef58-4346-90c9-6abcab412125/tuned/0.log"
Jul 29 11:06:14.456726 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:14.456691    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6748489656-hlxcf_6f242b84-732d-4642-b7a9-8e6c874d0bdd/cluster-samples-operator/0.log"
Jul 29 11:06:14.469627 ip-10-0-133-233 kubenswrapper[2564]: I0729 11:06:14.469593    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6748489656-hlxcf_6f242b84-732d-4642-b7a9-8e6c874d0bdd/cluster-samples-operator-watch/0.log"