Sep 09 20:50:23.532354 ip-10-0-129-24 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 09 20:50:23.532367 ip-10-0-129-24 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 09 20:50:23.532375 ip-10-0-129-24 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 09 20:50:23.532606 ip-10-0-129-24 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 09 20:50:33.781948 ip-10-0-129-24 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 09 20:50:33.781966 ip-10-0-129-24 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot fb7750fe8d244ccfa3d3344bd8e496ca --
Sep 09 20:52:28.229274 ip-10-0-129-24 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 20:52:28.568442 ip-10-0-129-24 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 20:52:28.568442 ip-10-0-129-24 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 20:52:28.568442 ip-10-0-129-24 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 20:52:28.568442 ip-10-0-129-24 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 20:52:28.568442 ip-10-0-129-24 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 20:52:28.570025 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.569932    2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 20:52:28.575489 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575457    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:52:28.575489 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575480    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:52:28.575489 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575484    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:52:28.575489 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575487    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:52:28.575489 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575490    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:52:28.575489 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575494    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:52:28.575489 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575498    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575502    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575505    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575508    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575511    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575515    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575518    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575520    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575523    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575526    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575528    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575531    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575534    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575536    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575538    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575541    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575544    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575546    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575549    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575551    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:52:28.575758 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575554    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575557    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575559    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575562    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575565    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575568    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575571    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575573    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575576    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575579    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575581    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575584    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575587    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575589    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575592    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575595    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575598    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575600    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575603    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:52:28.576262 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575605    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575608    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575610    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575613    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575616    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575619    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575621    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575624    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575627    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575629    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575632    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575634    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575637    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575643    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575647    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575650    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575652    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575656    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575660    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:52:28.576729 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575663    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575666    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575669    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575672    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575675    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575678    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575681    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575684    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575687    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575690    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575693    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575696    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575698    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575701    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575703    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575706    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575709    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575711    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575714    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575724    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:52:28.577211 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575727    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575729    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575732    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575735    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575737    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575740    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575743    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575747    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575750    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575753    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575756    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575759    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575762    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575765    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.575768    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576565    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576574    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576577    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576579    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576582    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:52:28.577707 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576585    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576587    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576590    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576593    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576596    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576598    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576601    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576603    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576606    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576608    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576611    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576613    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576620    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576624    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576627    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576630    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576632    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576635    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576638    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576641    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:52:28.578222 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576643    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576646    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576649    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576652    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576654    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576656    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576659    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576662    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576665    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576667    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576670    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576673    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576676    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576679    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576681    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576684    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576687    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576689    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576692    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576696    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:52:28.578723 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576698    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576701    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576703    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576706    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576709    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576712    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576714    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576717    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576719    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576722    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576725    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576727    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576730    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576733    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576736    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576739    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576742    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576744    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576747    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:52:28.579233 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576750    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576753    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576755    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576758    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576761    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576764    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576766    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576770    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576774    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576778    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576782    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576784    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576787    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576789    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576792    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576794    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576797    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576799    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576802    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576805    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:52:28.579716 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576808    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576810    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576813    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576816    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576819    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576821    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576824    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576826    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576829    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576832    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576834    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576837    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576840    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576843    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.576845    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577279    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577284    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577288    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577291    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577294    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:52:28.580224 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577297    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577300    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577302    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577305    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577308    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577310    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577313    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577316    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577319    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577321    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577324    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577327    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577330    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577333    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577335    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577338    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577340    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577344    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577347    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577349    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:52:28.580700 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577352    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577355    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577358    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577360    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577363    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577366    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577369    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577371    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577374    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577378    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577382    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577384    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577387    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577390    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577393    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577395    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577398    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577400    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577403    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:52:28.581241 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577405    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577408    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577412    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577415    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577418    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577422    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577426    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577428    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577431    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577433    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577436    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577438    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577441    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577444    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577446    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577449    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577452    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577454    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577457    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:52:28.581724 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577459    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577462    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577464    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577467    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577470    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577472    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577475    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577478    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577481    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577483    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577486    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577488    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577491    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577493    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577496    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577498    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577501    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577503    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577506    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577509    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:52:28.582215 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577512    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577514    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577518    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577521    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577523    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577526    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577529    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577531    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577535    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577538    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577541    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577543    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577546    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577548    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577551    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.577554    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577627    2585 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577665    2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577674    2585 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577679    2585 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577683    2585 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 20:52:28.582731 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577686    2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577691    2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577696    2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577699    2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577702    2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577705    2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577708    2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577712    2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577715    2585 flags.go:64] FLAG: --cgroup-root=""
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577718    2585 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577721    2585 flags.go:64] FLAG: --client-ca-file=""
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577726    2585 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577729    2585 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577733    2585 flags.go:64] FLAG: --cluster-domain=""
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577736    2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577739    2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577742    2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577745    2585 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577750    2585 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577753    2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577756    2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577759    2585 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577762    2585 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577765    2585 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 20:52:28.583271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577776    2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577780    2585 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577783    2585 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577787    2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577790    2585 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577794    2585 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577797    2585 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577800    2585 flags.go:64] FLAG: --enable-server="true"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577803    2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577808    2585 flags.go:64] FLAG: --event-burst="100"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577811    2585 flags.go:64] FLAG: --event-qps="50"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577814    2585 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577817    2585 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577820    2585 flags.go:64] FLAG: --eviction-hard=""
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577824    2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577827    2585 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577829    2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577832    2585 flags.go:64] FLAG: --eviction-soft=""
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577835    2585 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577838    2585 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577842    2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577845    2585 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577848    2585 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577850    2585 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577853    2585 flags.go:64] FLAG: --feature-gates=""
Sep 09 20:52:28.583856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577858    2585 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577861    2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577865    2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577867    2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577870    2585 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577874    2585 flags.go:64] FLAG: --help="false"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577893    2585 flags.go:64] FLAG: --hostname-override="ip-10-0-129-24.ec2.internal"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577897    2585 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577900    2585 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577903    2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577906    2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577910    2585 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577914    2585 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577917    2585 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577920    2585 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577923    2585 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577926    2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577929    2585 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577932    2585 flags.go:64] FLAG: --kube-reserved=""
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577934    2585 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577937    2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577940    2585 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577943    2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577946    2585 flags.go:64] FLAG: --lock-file=""
Sep 09 20:52:28.584517 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577949    2585 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577952    2585 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577955    2585 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577961    2585 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577964    2585 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577967    2585 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577970    2585 flags.go:64] FLAG: --logging-format="text"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577973    2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577977    2585 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577980    2585 flags.go:64] FLAG: --manifest-url=""
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577983    2585 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577987    2585 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577990    2585 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577994    2585 flags.go:64] FLAG: --max-pods="110"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.577997    2585 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578000    2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578003    2585 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578006    2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578009    2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578012    2585 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578016    2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578024    2585 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578027    2585 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578030    2585 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 20:52:28.585122 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578033    2585 flags.go:64] FLAG: --pod-cidr=""
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578036    2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578043    2585 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578046    2585 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578049    2585 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578052    2585 flags.go:64] FLAG: --port="10250"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578055    2585 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578058    2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08a6e6245c521be47"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578061    2585 flags.go:64] FLAG: --qos-reserved=""
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578064    2585 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578067    2585 flags.go:64] FLAG: --register-node="true"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578070    2585 flags.go:64] FLAG: --register-with-taints=""
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578073    2585 flags.go:64] FLAG: --registry-burst="10"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578077    2585 flags.go:64] FLAG: --registry-qps="5"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578080    2585 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578083    2585 flags.go:64] FLAG: --reserved-memory=""
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578086    2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578091    2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578094    2585 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578097    2585 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578100    2585 flags.go:64] FLAG: --runonce="false"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578103    2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578106    2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578108    2585 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578111    2585 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 20:52:28.585713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578114    2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578117    2585 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578121    2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578125    2585 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578128    2585 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578131    2585 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578134    2585 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578136    2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578139    2585 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578142    2585 flags.go:64] FLAG: --system-cgroups=""
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578145    2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578150    2585 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578153    2585 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578156    2585 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578160    2585 flags.go:64] FLAG: --tls-min-version=""
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578163    2585 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578166    2585 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578169    2585 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578171    2585 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578174    2585 flags.go:64] FLAG: --v="2"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578179    2585 flags.go:64] FLAG: --version="false"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578184    2585 flags.go:64] FLAG: --vmodule=""
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578188    2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578191    2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578321    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:52:28.586477 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578324    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578328    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578330    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578333    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578335    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578338    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578341    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578343    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578346    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578349    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578353    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578356    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578358    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578361    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578365    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578369    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578372    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578375    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578378    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:52:28.587200 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578381    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578384    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578387    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578390    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578392    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578395    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578398    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578400    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578403    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578406    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578412    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578414    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578418    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578421    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578424    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578426    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578429    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578431    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578434    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578436    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:52:28.587686 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578439    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578441    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578444    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578448    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578452    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578455    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578458    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578460    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578463    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578466    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578469    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578472    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578475    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578477    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578480    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578483    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578487    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578490    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578493    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:52:28.588185 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578496    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578498    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578501    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578503    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578506    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578510    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578512    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578515    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578517    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578520    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578522    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578525    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578528    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578530    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578533    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578535    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578539    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578542    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578544    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578547    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:52:28.588667 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578549    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578552    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578555    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578557    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578559    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578562    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578564    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578567    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578569    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578572    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578574    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578577    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578580    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578582    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578585    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578588    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578591    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578598    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578600    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:52:28.589169 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.578603    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:52:28.589623 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.578608    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 20:52:28.589623 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.586044    2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 20:52:28.589623 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.586182    2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 20:52:28.589623 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.587010    2585 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 20:52:28.589623 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.587018    2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 20:52:28.589623 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587081    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:52:28.589623 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587086    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:52:28.589623 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587090    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:52:28.589623 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587093    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:52:28.589623 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587096    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:52:28.589623 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587099    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:52:28.589623 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587102    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:52:28.589623 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587105    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:52:28.589623 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587108    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:52:28.589623 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587111    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587114    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587118    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587121    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587123    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587126    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587130    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587132    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587135    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587138    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587141    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587144    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587147    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587150    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587152    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587155    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587158    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587162    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587165    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:52:28.590067 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587168    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587172    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587175    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587177    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587180    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587183    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587186    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587188    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587191    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587195    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587199    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587202    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587205    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587207    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587210    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587213    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587215    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587218    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587221    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:52:28.590543 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587223    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587226    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587228    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587231    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587234    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587237    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587239    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587242    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587245    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587247    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587250    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587252    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587255    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587258    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587261    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587264    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587266    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587269    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587271    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587274    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:52:28.591048 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587276    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587279    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587281    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587284    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587286    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587288    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587291    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587295    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587298    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587301    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587304    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587306    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587310    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587313    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587315    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587318    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587321    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587324    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587327    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587329    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:52:28.591536 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587331    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:52:28.592041 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587334    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:52:28.592041 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587336    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:52:28.592041 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587339    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:52:28.592041 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587342    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:52:28.592041 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587345    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:52:28.592041 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587347    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:52:28.592041 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587350    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:52:28.592041 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587353    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:52:28.592041 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587356    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:52:28.592041 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587359    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:52:28.592041 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587362    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:52:28.592041 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.587367    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 20:52:28.592041 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587482    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:52:28.592041 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587487    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:52:28.592041 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587490    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587493    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587496    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587500    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587505    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587507    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587510    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587513    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587516    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587519    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587521    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587524    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587526    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587529    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587532    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587535    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587538    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587541    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587544    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:52:28.592418 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587547    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587549    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587552    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587554    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587558    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587560    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587563    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587567    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587569    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587572    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587574    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587577    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587580    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587582    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587585    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587587    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587590    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587592    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587595    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587597    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:52:28.592916 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587600    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587602    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587605    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587607    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587610    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587612    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587615    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587617    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587620    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587622    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587625    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587628    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587631    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587633    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587636    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587639    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587641    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587644    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587647    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587649    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:52:28.593411 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587652    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587655    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587658    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587660    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587662    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587665    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587668    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587670    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587673    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587675    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587679    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587682    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587685    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587687    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587690    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587693    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587695    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587698    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587701    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587704    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:52:28.593909 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587706    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587709    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587712    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587714    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587717    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587719    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587722    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587725    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587727    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587730    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587733    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587735    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587738    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587740    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587743    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587745    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587748    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:52:28.594395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:28.587750    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:52:28.594809 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.587755    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 20:52:28.594809 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.587872    2585 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 20:52:28.594809 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.589832    2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 20:52:28.594809 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.590572    2585 server.go:1019] "Starting client certificate rotation"
Sep 09 20:52:28.594809 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.590671    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 20:52:28.594809 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.590703    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 20:52:28.594809 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.592571    2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 20:52:28.597175 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.596370    2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 20:52:28.605198 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.605176    2585 log.go:25] "Validated CRI v1 runtime API"
Sep 09 20:52:28.608816 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.608800    2585 log.go:25] "Validated CRI v1 image API"
Sep 09 20:52:28.609710 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.609696    2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 20:52:28.615182 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.615163    2585 fs.go:135] Filesystem UUIDs: map[295b8446-10e8-4426-aac0-817165fc6c79:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9ccfb3bd-ef47-4a45-b826-573b2bbaba60:/dev/nvme0n1p3]
Sep 09 20:52:28.615249 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.615182    2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 20:52:28.619414 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.619298    2585 manager.go:216] Machine: {Timestamp:2026-09-09 20:52:28.618807542 +0000 UTC m=+0.289437233 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3138694 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28bbcbe8e7151f94a25882fb94cc4e SystemUUID:ec28bbcb-e8e7-151f-94a2-5882fb94cc4e BootID:fb7750fe-8d24-4ccf-a3d3-344bd8e496ca Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:56:91:a3:1a:cd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:56:91:a3:1a:cd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:1d:09:58:71:e4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 20:52:28.619414 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.619411    2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 20:52:28.619534 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.619522    2585 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 20:52:28.620489 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.620463    2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 20:52:28.620664 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.620492    2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-24.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 20:52:28.621621 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.621609    2585 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 20:52:28.621656 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.621622    2585 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 20:52:28.621656 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.621630    2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 20:52:28.622229 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.622217    2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 20:52:28.623350 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.623340    2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 20:52:28.627405 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.627392    2585 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 20:52:28.627489 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.627470    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 20:52:28.627528 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.627516    2585 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 20:52:28.627559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.627540    2585 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 20:52:28.627559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.627554    2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 20:52:28.627614 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.627571    2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 20:52:28.627614 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.627586    2585 kubelet.go:398] "Adding apiserver pod source"
Sep 09 20:52:28.627614 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.627596    2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 20:52:28.627993 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.627980    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 20:52:28.628033 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.628002    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 20:52:28.629998 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.629978    2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 20:52:28.631930 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.631915    2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 20:52:28.631987 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.631949    2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 20:52:28.632635 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.632623    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 20:52:28.632676 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.632641    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 20:52:28.632676 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.632647    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 20:52:28.632676 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.632652    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 20:52:28.632676 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.632660    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 20:52:28.632676 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.632665    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 20:52:28.632676 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.632679    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 20:52:28.632961 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.632685    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 20:52:28.632961 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.632697    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 20:52:28.632961 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.632703    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 20:52:28.632961 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.632716    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 20:52:28.632961 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.632722    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 20:52:28.632961 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.632742    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 20:52:28.632961 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.632747    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 20:52:28.636704 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.636689    2585 server.go:1268] "Started kubelet"
Sep 09 20:52:28.636794 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.636747    2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 20:52:28.636864 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.636789    2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 20:52:28.636864 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.636855    2585 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 20:52:28.640203 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.640175    2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 20:52:28.640345 ip-10-0-129-24 systemd[1]: Started Kubernetes Kubelet.
Sep 09 20:52:28.640481 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.640398    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 20:52:28.640597 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.640457    2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-24.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 20:52:28.641600 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.640819    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-24.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 20:52:28.643925 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.643095    2585 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 20:52:28.643925 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.643391    2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 20:52:28.646275 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.646254    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 20:52:28.646709 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.646690    2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 20:52:28.646864 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.645675    2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-24.ec2.internal.18d3c2554e3a3221  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-24.ec2.internal,UID:ip-10-0-129-24.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-24.ec2.internal,},FirstTimestamp:2026-09-09 20:52:28.636664353 +0000 UTC m=+0.307294043,LastTimestamp:2026-09-09 20:52:28.636664353 +0000 UTC m=+0.307294043,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-24.ec2.internal,}"
Sep 09 20:52:28.646962 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.646899    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-24.ec2.internal\" not found"
Sep 09 20:52:28.647327 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.647309    2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 20:52:28.647410 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.647323    2585 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 20:52:28.647410 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.647367    2585 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 20:52:28.647625 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.647456    2585 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 20:52:28.647625 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.647465    2585 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 20:52:28.648051 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.648021    2585 factory.go:153] Registering CRI-O factory
Sep 09 20:52:28.648051 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.648048    2585 factory.go:223] Registration of the crio container factory successfully
Sep 09 20:52:28.648221 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.648092    2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 20:52:28.648221 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.648099    2585 factory.go:55] Registering systemd factory
Sep 09 20:52:28.648221 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.648104    2585 factory.go:223] Registration of the systemd container factory successfully
Sep 09 20:52:28.648221 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.648118    2585 factory.go:103] Registering Raw factory
Sep 09 20:52:28.648221 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.648128    2585 manager.go:1196] Started watching for new ooms in manager
Sep 09 20:52:28.648893 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.648866    2585 manager.go:349] Starting recovery of all containers
Sep 09 20:52:28.660740 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.660720    2585 manager.go:354] Recovery completed
Sep 09 20:52:28.661036 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.661007    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 09 20:52:28.661115 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.661045    2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-24.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 09 20:52:28.665245 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.665231    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:52:28.667087 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.667073    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:52:28.667158 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.667104    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:52:28.667158 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.667120    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:52:28.667915 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.667899    2585 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 20:52:28.667915 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.667915    2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 20:52:28.668051 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.667950    2585 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 20:52:28.668051 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.667905    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:52:28.668779 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.668764    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:52:28.668842 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.668789    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:52:28.668842 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.668800    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:52:28.670108 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.670090    2585 policy_none.go:49] "None policy: Start"
Sep 09 20:52:28.670174 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.670118    2585 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 20:52:28.670174 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.670134    2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 20:52:28.670174 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.670168    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4whng"
Sep 09 20:52:28.672065 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.672050    2585 policy_none.go:47] "Start"
Sep 09 20:52:28.678148 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.678040    2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-24.ec2.internal.18d3c255500a6b9b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-24.ec2.internal,UID:ip-10-0-129-24.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-24.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-24.ec2.internal,},FirstTimestamp:2026-09-09 20:52:28.667087771 +0000 UTC m=+0.337717463,LastTimestamp:2026-09-09 20:52:28.667087771 +0000 UTC m=+0.337717463,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-24.ec2.internal,}"
Sep 09 20:52:28.679762 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.679740    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4whng"
Sep 09 20:52:28.697635 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.697614    2585 manager.go:337] "Starting Device Plugin manager"
Sep 09 20:52:28.725505 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.697664    2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 20:52:28.725505 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.697679    2585 server.go:87] "Starting device plugin registration server"
Sep 09 20:52:28.725505 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.698096    2585 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 20:52:28.725505 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.698108    2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 20:52:28.725505 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.698249    2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 20:52:28.725505 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.698337    2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 20:52:28.725505 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.698345    2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 20:52:28.725505 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.698921    2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 20:52:28.725505 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.698980    2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-24.ec2.internal\" not found"
Sep 09 20:52:28.791421 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.791376    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 20:52:28.792650 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.792629    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 20:52:28.792730 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.792688    2585 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 20:52:28.792730 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.792713    2585 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 20:52:28.792807 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.792762    2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 09 20:52:28.795396 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.795240    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:52:28.798688 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.798671    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:52:28.799568 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.799554    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:52:28.799635 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.799582    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:52:28.799635 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.799595    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:52:28.799635 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.799618    2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-24.ec2.internal"
Sep 09 20:52:28.806803 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.806784    2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-24.ec2.internal"
Sep 09 20:52:28.806887 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.806815    2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-24.ec2.internal\": node \"ip-10-0-129-24.ec2.internal\" not found"
Sep 09 20:52:28.828669 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.828593    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-24.ec2.internal\" not found"
Sep 09 20:52:28.893046 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.892995    2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-24.ec2.internal"]
Sep 09 20:52:28.893171 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.893103    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:52:28.895105 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.895087    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:52:28.895194 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.895117    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:52:28.895194 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.895127    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:52:28.897343 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.897329    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:52:28.897500 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.897473    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal"
Sep 09 20:52:28.897541 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.897518    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:52:28.898219 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.898200    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:52:28.898302 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.898235    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:52:28.898302 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.898245    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:52:28.898302 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.898200    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:52:28.898403 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.898318    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:52:28.898403 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.898331    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:52:28.900291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.900277    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-24.ec2.internal"
Sep 09 20:52:28.900360 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.900304    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:52:28.901171 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.901154    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:52:28.901220 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.901191    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:52:28.901220 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.901207    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:52:28.922770 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.922740    2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-24.ec2.internal\" not found" node="ip-10-0-129-24.ec2.internal"
Sep 09 20:52:28.927218 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.927198    2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-24.ec2.internal\" not found" node="ip-10-0-129-24.ec2.internal"
Sep 09 20:52:28.929249 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:28.929232    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-24.ec2.internal\" not found"
Sep 09 20:52:28.948789 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.948760    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee33b1848fc9d62f869c7f00a9a00592-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal\" (UID: \"ee33b1848fc9d62f869c7f00a9a00592\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal"
Sep 09 20:52:28.948872 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.948796    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee33b1848fc9d62f869c7f00a9a00592-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal\" (UID: \"ee33b1848fc9d62f869c7f00a9a00592\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal"
Sep 09 20:52:28.948872 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:28.948815    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ad114d159ddf9784199c66d4e552938-config\") pod \"kube-apiserver-proxy-ip-10-0-129-24.ec2.internal\" (UID: \"4ad114d159ddf9784199c66d4e552938\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-24.ec2.internal"
Sep 09 20:52:29.029348 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:29.029319    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-24.ec2.internal\" not found"
Sep 09 20:52:29.049126 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.049101    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee33b1848fc9d62f869c7f00a9a00592-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal\" (UID: \"ee33b1848fc9d62f869c7f00a9a00592\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal"
Sep 09 20:52:29.049186 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.049131    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee33b1848fc9d62f869c7f00a9a00592-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal\" (UID: \"ee33b1848fc9d62f869c7f00a9a00592\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal"
Sep 09 20:52:29.049186 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.049156    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ad114d159ddf9784199c66d4e552938-config\") pod \"kube-apiserver-proxy-ip-10-0-129-24.ec2.internal\" (UID: \"4ad114d159ddf9784199c66d4e552938\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-24.ec2.internal"
Sep 09 20:52:29.049256 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.049223    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee33b1848fc9d62f869c7f00a9a00592-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal\" (UID: \"ee33b1848fc9d62f869c7f00a9a00592\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal"
Sep 09 20:52:29.049289 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.049279    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ad114d159ddf9784199c66d4e552938-config\") pod \"kube-apiserver-proxy-ip-10-0-129-24.ec2.internal\" (UID: \"4ad114d159ddf9784199c66d4e552938\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-24.ec2.internal"
Sep 09 20:52:29.049322 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.049311    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee33b1848fc9d62f869c7f00a9a00592-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal\" (UID: \"ee33b1848fc9d62f869c7f00a9a00592\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal"
Sep 09 20:52:29.129550 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:29.129476    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-24.ec2.internal\" not found"
Sep 09 20:52:29.224992 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.224950    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal"
Sep 09 20:52:29.230381 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:29.230082    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-24.ec2.internal\" not found"
Sep 09 20:52:29.230381 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.230095    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-24.ec2.internal"
Sep 09 20:52:29.330892 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:29.330841    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-24.ec2.internal\" not found"
Sep 09 20:52:29.436228 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:29.436150    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-24.ec2.internal\" not found"
Sep 09 20:52:29.511643 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.511616    2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:52:29.537443 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:29.537407    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-24.ec2.internal\" not found"
Sep 09 20:52:29.593098 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.593062    2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 20:52:29.593614 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.593270    2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 20:52:29.593614 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.593291    2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 20:52:29.637703 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:29.637658    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-24.ec2.internal\" not found"
Sep 09 20:52:29.683246 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.683210    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 20:47:29 +0000 UTC" deadline="2028-03-15 19:45:34.938705463 +0000 UTC"
Sep 09 20:52:29.683246 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.683241    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13270h53m5.255469237s"
Sep 09 20:52:29.719346 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.719254    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:52:29.720375 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.720356    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:52:29.720467 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.720390    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:52:29.720467 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.720403    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:52:29.738710 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:29.738681    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-24.ec2.internal\" not found"
Sep 09 20:52:29.839258 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:29.839222    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-24.ec2.internal\" not found"
Sep 09 20:52:29.893818 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.893787    2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:52:29.909202 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:29.909012    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ad114d159ddf9784199c66d4e552938.slice/crio-c198e27babbfaf4342e9f884f2e94266b4c5332b17ad3930058ceeaa529d3de8 WatchSource:0}: Error finding container c198e27babbfaf4342e9f884f2e94266b4c5332b17ad3930058ceeaa529d3de8: Status 404 returned error can't find the container with id c198e27babbfaf4342e9f884f2e94266b4c5332b17ad3930058ceeaa529d3de8
Sep 09 20:52:29.914362 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:29.914344    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 20:52:29.940332 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:29.940303    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-24.ec2.internal\" not found"
Sep 09 20:52:30.044093 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:30.044056    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-24.ec2.internal\" not found"
Sep 09 20:52:30.059564 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.059539    2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:52:30.147721 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.147692    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal"
Sep 09 20:52:30.162184 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.162161    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 20:52:30.163244 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.163228    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-24.ec2.internal"
Sep 09 20:52:30.176968 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.176951    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 20:52:30.326415 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:30.326384    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee33b1848fc9d62f869c7f00a9a00592.slice/crio-58265c59a9b26f10db4a858b4cf958d0a9369191a0b4aa8c45798ed2d68f7d9f WatchSource:0}: Error finding container 58265c59a9b26f10db4a858b4cf958d0a9369191a0b4aa8c45798ed2d68f7d9f: Status 404 returned error can't find the container with id 58265c59a9b26f10db4a858b4cf958d0a9369191a0b4aa8c45798ed2d68f7d9f
Sep 09 20:52:30.630321 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.630219    2585 apiserver.go:52] "Watching apiserver"
Sep 09 20:52:30.641995 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.641960    2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 20:52:30.643088 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.643053    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-x5c85","openshift-multus/network-metrics-daemon-ltd4k","openshift-network-operator/iptables-alerter-9pczp","kube-system/kube-apiserver-proxy-ip-10-0-129-24.ec2.internal","openshift-cluster-node-tuning-operator/tuned-w5btq","openshift-dns/node-resolver-gvwv2","openshift-multus/multus-additional-cni-plugins-pwq6l","openshift-network-diagnostics/network-check-target-4ct29","openshift-ovn-kubernetes/ovnkube-node-82b4g","kube-system/global-pull-secret-syncer-c28zc","kube-system/konnectivity-agent-pb6hf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q","openshift-image-registry/node-ca-kgvvq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal"]
Sep 09 20:52:30.646015 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.645987    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:30.646146 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:30.646106    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:52:30.647719 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.647701    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 20:52:30.648492 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.648463    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kgvvq"
Sep 09 20:52:30.650847 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.650801    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9pczp"
Sep 09 20:52:30.651470 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.651445    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 20:52:30.651621 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.651447    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xhx7d\""
Sep 09 20:52:30.652085 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.652063    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 20:52:30.652187 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.652121    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 20:52:30.653210 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.653184    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:30.653307 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:30.653276    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:52:30.653944 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.653923    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kh2p6\""
Sep 09 20:52:30.654856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.654836    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:52:30.654962 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.654874    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 20:52:30.655788 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.655754    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 20:52:30.656035 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.656020    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.657855 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.657821    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-kubelet-config\") pod \"global-pull-secret-syncer-c28zc\" (UID: \"95b5dc36-8d5d-43d2-9444-5e870b1ca94b\") " pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:30.657980 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.657872    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-dbus\") pod \"global-pull-secret-syncer-c28zc\" (UID: \"95b5dc36-8d5d-43d2-9444-5e870b1ca94b\") " pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:30.657980 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.657900    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f399b85-c8bd-417f-ae6a-1085b82a4a58-host\") pod \"node-ca-kgvvq\" (UID: \"3f399b85-c8bd-417f-ae6a-1085b82a4a58\") " pod="openshift-image-registry/node-ca-kgvvq"
Sep 09 20:52:30.657980 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.657925    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret\") pod \"global-pull-secret-syncer-c28zc\" (UID: \"95b5dc36-8d5d-43d2-9444-5e870b1ca94b\") " pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:30.658120 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.657977    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f399b85-c8bd-417f-ae6a-1085b82a4a58-serviceca\") pod \"node-ca-kgvvq\" (UID: \"3f399b85-c8bd-417f-ae6a-1085b82a4a58\") " pod="openshift-image-registry/node-ca-kgvvq"
Sep 09 20:52:30.658120 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.658045    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9lg7\" (UniqueName: \"kubernetes.io/projected/3f399b85-c8bd-417f-ae6a-1085b82a4a58-kube-api-access-m9lg7\") pod \"node-ca-kgvvq\" (UID: \"3f399b85-c8bd-417f-ae6a-1085b82a4a58\") " pod="openshift-image-registry/node-ca-kgvvq"
Sep 09 20:52:30.658988 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.658964    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gvwv2"
Sep 09 20:52:30.659605 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.659586    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-z4gpk\""
Sep 09 20:52:30.659699 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.659659    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:52:30.659785 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.659764    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 20:52:30.661480 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.661461    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.662822 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.662799    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 20:52:30.662822 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.662817    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 20:52:30.663026 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.663007    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-82cx7\""
Sep 09 20:52:30.663765 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.663747    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.665952 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.665933    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qbnxx\""
Sep 09 20:52:30.666109 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.665972    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 20:52:30.666498 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.666478    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:30.666712 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:30.666690    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:52:30.669178 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.669152    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.670172 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.670154    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 20:52:30.670483 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.670460    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 20:52:30.671595 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.671563    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 20:52:30.671705 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.671667    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 20:52:30.671785 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.671768    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 20:52:30.672000 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.671980    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 20:52:30.672500 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.672393    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pb6hf"
Sep 09 20:52:30.675370 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.675033    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 20:52:30.675370 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.675094    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 20:52:30.675370 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.675151    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4q2w5\""
Sep 09 20:52:30.675370 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.675316    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 20:52:30.675644 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.675458    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 20:52:30.675644 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.675564    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 20:52:30.675725 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.675663    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rt9mm\""
Sep 09 20:52:30.675789 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.675763    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 20:52:30.676732 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.676712    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.677041 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.677019    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-prc5b\""
Sep 09 20:52:30.677325 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.677306    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 20:52:30.677403 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.677327    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 20:52:30.681460 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.681437    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 20:52:30.682178 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.681738    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 20:52:30.682178 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.681936    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9sfcq\""
Sep 09 20:52:30.682178 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.682068    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 20:52:30.696023 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.696002    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5wb6q"
Sep 09 20:52:30.706320 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.706295    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5wb6q"
Sep 09 20:52:30.748922 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.748902    2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 20:52:30.758215 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758185    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-var-lib-cni-multus\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.758327 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758229    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-etc-kubernetes\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.758327 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758261    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret\") pod \"global-pull-secret-syncer-c28zc\" (UID: \"95b5dc36-8d5d-43d2-9444-5e870b1ca94b\") " pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:30.758327 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758287    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.758327 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758314    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0c1af91b-e37d-4a36-a741-21f2bcb399ac-iptables-alerter-script\") pod \"iptables-alerter-9pczp\" (UID: \"0c1af91b-e37d-4a36-a741-21f2bcb399ac\") " pod="openshift-network-operator/iptables-alerter-9pczp"
Sep 09 20:52:30.758563 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758338    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c1af91b-e37d-4a36-a741-21f2bcb399ac-host-slash\") pod \"iptables-alerter-9pczp\" (UID: \"0c1af91b-e37d-4a36-a741-21f2bcb399ac\") " pod="openshift-network-operator/iptables-alerter-9pczp"
Sep 09 20:52:30.758563 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:30.758375    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:52:30.758563 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758379    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-sysctl-d\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.758563 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:30.758453    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret podName:95b5dc36-8d5d-43d2-9444-5e870b1ca94b nodeName:}" failed. No retries permitted until 2026-09-09 20:52:31.258419956 +0000 UTC m=+2.929049634 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret") pod "global-pull-secret-syncer-c28zc" (UID: "95b5dc36-8d5d-43d2-9444-5e870b1ca94b") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:52:30.758563 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758479    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-run-openvswitch\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.758563 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758515    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ffdae73-5e32-418c-8138-171b99a0c118-multus-daemon-config\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.758563 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758545    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f399b85-c8bd-417f-ae6a-1085b82a4a58-host\") pod \"node-ca-kgvvq\" (UID: \"3f399b85-c8bd-417f-ae6a-1085b82a4a58\") " pod="openshift-image-registry/node-ca-kgvvq"
Sep 09 20:52:30.758934 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758567    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-systemd-units\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.758934 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758585    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/59d9f229-3224-476f-ae39-e8b4e039bae5-agent-certs\") pod \"konnectivity-agent-pb6hf\" (UID: \"59d9f229-3224-476f-ae39-e8b4e039bae5\") " pod="kube-system/konnectivity-agent-pb6hf"
Sep 09 20:52:30.758934 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758599    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f665bfb-5209-4095-9276-31d40e0d326b-cni-binary-copy\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.758934 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758642    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-var-lib-cni-bin\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.758934 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758692    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f399b85-c8bd-417f-ae6a-1085b82a4a58-host\") pod \"node-ca-kgvvq\" (UID: \"3f399b85-c8bd-417f-ae6a-1085b82a4a58\") " pod="openshift-image-registry/node-ca-kgvvq"
Sep 09 20:52:30.758934 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758747    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxvfx\" (UniqueName: \"kubernetes.io/projected/1ffdae73-5e32-418c-8138-171b99a0c118-kube-api-access-jxvfx\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.758934 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758777    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-sysctl-conf\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.758934 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758802    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f5245560-79ba-41c7-9676-d6278d5b8675-etc-tuned\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.758934 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758840    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-multus-cni-dir\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.758934 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758879    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-etc-selinux\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.759381 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.758978    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-modprobe-d\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.759381 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759008    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-var-lib-openvswitch\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.759381 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759032    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c01881d-abb4-462f-8807-43242a0e0dbd-ovnkube-config\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.759381 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759058    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-hostroot\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.759381 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759107    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9lg7\" (UniqueName: \"kubernetes.io/projected/3f399b85-c8bd-417f-ae6a-1085b82a4a58-kube-api-access-m9lg7\") pod \"node-ca-kgvvq\" (UID: \"3f399b85-c8bd-417f-ae6a-1085b82a4a58\") " pod="openshift-image-registry/node-ca-kgvvq"
Sep 09 20:52:30.759381 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759162    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqnj5\" (UniqueName: \"kubernetes.io/projected/d23607fd-429c-4a38-acc1-fbac3cd34147-kube-api-access-bqnj5\") pod \"node-resolver-gvwv2\" (UID: \"d23607fd-429c-4a38-acc1-fbac3cd34147\") " pod="openshift-dns/node-resolver-gvwv2"
Sep 09 20:52:30.759381 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759201    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-kubelet\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.759381 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759230    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c01881d-abb4-462f-8807-43242a0e0dbd-ovn-node-metrics-cert\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.759381 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759263    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-socket-dir\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.759381 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759295    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-sys-fs\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.759381 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759322    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-systemd\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.759381 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759344    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-sys\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.759381 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759370    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f665bfb-5209-4095-9276-31d40e0d326b-cnibin\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.759905 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759393    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-run-netns\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.759905 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759425    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-lib-modules\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.759905 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759449    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-host\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.759905 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759474    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-cni-bin\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.759905 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759497    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f665bfb-5209-4095-9276-31d40e0d326b-system-cni-dir\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.759905 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759535    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-cnibin\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.759905 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759551    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-run-multus-certs\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.759905 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759581    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-kubernetes\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.759905 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759614    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-node-log\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.759905 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759636    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c01881d-abb4-462f-8807-43242a0e0dbd-env-overrides\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.759905 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759669    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs\") pod \"network-metrics-daemon-ltd4k\" (UID: \"c907cd72-cf97-4133-87f8-2e9bb7fb67a6\") " pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:30.759905 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759698    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjrkr\" (UniqueName: \"kubernetes.io/projected/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-kube-api-access-cjrkr\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.759905 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759723    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g98fg\" (UniqueName: \"kubernetes.io/projected/0c1af91b-e37d-4a36-a741-21f2bcb399ac-kube-api-access-g98fg\") pod \"iptables-alerter-9pczp\" (UID: \"0c1af91b-e37d-4a36-a741-21f2bcb399ac\") " pod="openshift-network-operator/iptables-alerter-9pczp"
Sep 09 20:52:30.759905 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759753    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-log-socket\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.759905 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759773    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-registration-dir\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.759905 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759796    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-device-dir\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.760559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759818    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgbmn\" (UniqueName: \"kubernetes.io/projected/f5245560-79ba-41c7-9676-d6278d5b8675-kube-api-access-lgbmn\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.760559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759835    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d23607fd-429c-4a38-acc1-fbac3cd34147-hosts-file\") pod \"node-resolver-gvwv2\" (UID: \"d23607fd-429c-4a38-acc1-fbac3cd34147\") " pod="openshift-dns/node-resolver-gvwv2"
Sep 09 20:52:30.760559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759859    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3f665bfb-5209-4095-9276-31d40e0d326b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.760559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759880    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-slash\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.760559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759922    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.760559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.759968    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f665bfb-5209-4095-9276-31d40e0d326b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.760559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760002    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3f665bfb-5209-4095-9276-31d40e0d326b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.760559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760028    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-multus-socket-dir-parent\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.760559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760056    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-dbus\") pod \"global-pull-secret-syncer-c28zc\" (UID: \"95b5dc36-8d5d-43d2-9444-5e870b1ca94b\") " pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:30.760559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760080    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-run-ovn\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.760559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760103    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-cni-netd\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.760559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760127    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ffdae73-5e32-418c-8138-171b99a0c118-cni-binary-copy\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.760559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760148    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5245560-79ba-41c7-9676-d6278d5b8675-tmp\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.760559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760171    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-run-netns\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.760559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760195    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/59d9f229-3224-476f-ae39-e8b4e039bae5-konnectivity-ca\") pod \"konnectivity-agent-pb6hf\" (UID: \"59d9f229-3224-476f-ae39-e8b4e039bae5\") " pod="kube-system/konnectivity-agent-pb6hf"
Sep 09 20:52:30.760559 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760198    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-dbus\") pod \"global-pull-secret-syncer-c28zc\" (UID: \"95b5dc36-8d5d-43d2-9444-5e870b1ca94b\") " pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:30.761129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760221    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c74bg\" (UniqueName: \"kubernetes.io/projected/3f665bfb-5209-4095-9276-31d40e0d326b-kube-api-access-c74bg\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.761129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760246    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-multus-conf-dir\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.761129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760284    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-sysconfig\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.761129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760318    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-var-lib-kubelet\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.761129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760346    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f399b85-c8bd-417f-ae6a-1085b82a4a58-serviceca\") pod \"node-ca-kgvvq\" (UID: \"3f399b85-c8bd-417f-ae6a-1085b82a4a58\") " pod="openshift-image-registry/node-ca-kgvvq"
Sep 09 20:52:30.761129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760369    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-var-lib-kubelet\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.761129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760394    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-run-systemd\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.761129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760420    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-run-ovn-kubernetes\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.761129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760446    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8bjl\" (UniqueName: \"kubernetes.io/projected/9c01881d-abb4-462f-8807-43242a0e0dbd-kube-api-access-v8bjl\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.761129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760471    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6rp9\" (UniqueName: \"kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9\") pod \"network-check-target-4ct29\" (UID: \"04fa9846-4123-477d-8c86-481d721ecc1e\") " pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:30.761129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760497    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l9ds\" (UniqueName: \"kubernetes.io/projected/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-kube-api-access-5l9ds\") pod \"network-metrics-daemon-ltd4k\" (UID: \"c907cd72-cf97-4133-87f8-2e9bb7fb67a6\") " pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:30.761129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760543    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-run-k8s-cni-cncf-io\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.761129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760573    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-kubelet-config\") pod \"global-pull-secret-syncer-c28zc\" (UID: \"95b5dc36-8d5d-43d2-9444-5e870b1ca94b\") " pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:30.761129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760616    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-kubelet-config\") pod \"global-pull-secret-syncer-c28zc\" (UID: \"95b5dc36-8d5d-43d2-9444-5e870b1ca94b\") " pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:30.761129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760641    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-run\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.761129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760677    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d23607fd-429c-4a38-acc1-fbac3cd34147-tmp-dir\") pod \"node-resolver-gvwv2\" (UID: \"d23607fd-429c-4a38-acc1-fbac3cd34147\") " pod="openshift-dns/node-resolver-gvwv2"
Sep 09 20:52:30.761801 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760710    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-etc-openvswitch\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.761801 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760728    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c01881d-abb4-462f-8807-43242a0e0dbd-ovnkube-script-lib\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.761801 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.760754    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-system-cni-dir\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.761801 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.761256    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f399b85-c8bd-417f-ae6a-1085b82a4a58-serviceca\") pod \"node-ca-kgvvq\" (UID: \"3f399b85-c8bd-417f-ae6a-1085b82a4a58\") " pod="openshift-image-registry/node-ca-kgvvq"
Sep 09 20:52:30.774567 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.774541    2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 20:52:30.781107 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.781082    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9lg7\" (UniqueName: \"kubernetes.io/projected/3f399b85-c8bd-417f-ae6a-1085b82a4a58-kube-api-access-m9lg7\") pod \"node-ca-kgvvq\" (UID: \"3f399b85-c8bd-417f-ae6a-1085b82a4a58\") " pod="openshift-image-registry/node-ca-kgvvq"
Sep 09 20:52:30.799542 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.799491    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal" event={"ID":"ee33b1848fc9d62f869c7f00a9a00592","Type":"ContainerStarted","Data":"58265c59a9b26f10db4a858b4cf958d0a9369191a0b4aa8c45798ed2d68f7d9f"}
Sep 09 20:52:30.801057 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.801028    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-24.ec2.internal" event={"ID":"4ad114d159ddf9784199c66d4e552938","Type":"ContainerStarted","Data":"c198e27babbfaf4342e9f884f2e94266b4c5332b17ad3930058ceeaa529d3de8"}
Sep 09 20:52:30.861456 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861418    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-var-lib-kubelet\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.861456 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861464    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-run-systemd\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.861731 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861492    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-run-ovn-kubernetes\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.861731 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861549    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-run-ovn-kubernetes\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.861731 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861561    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-run-systemd\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.861731 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861566    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-var-lib-kubelet\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.861731 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861593    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8bjl\" (UniqueName: \"kubernetes.io/projected/9c01881d-abb4-462f-8807-43242a0e0dbd-kube-api-access-v8bjl\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.861731 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861635    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6rp9\" (UniqueName: \"kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9\") pod \"network-check-target-4ct29\" (UID: \"04fa9846-4123-477d-8c86-481d721ecc1e\") " pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:30.861731 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861659    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l9ds\" (UniqueName: \"kubernetes.io/projected/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-kube-api-access-5l9ds\") pod \"network-metrics-daemon-ltd4k\" (UID: \"c907cd72-cf97-4133-87f8-2e9bb7fb67a6\") " pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:30.861731 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861687    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-run-k8s-cni-cncf-io\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.861731 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861715    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-run\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861740    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d23607fd-429c-4a38-acc1-fbac3cd34147-tmp-dir\") pod \"node-resolver-gvwv2\" (UID: \"d23607fd-429c-4a38-acc1-fbac3cd34147\") " pod="openshift-dns/node-resolver-gvwv2"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861765    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-etc-openvswitch\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861797    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c01881d-abb4-462f-8807-43242a0e0dbd-ovnkube-script-lib\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861823    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-system-cni-dir\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861847    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-var-lib-cni-multus\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861867    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-run\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861875    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-etc-kubernetes\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861916    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-etc-kubernetes\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861919    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-run-k8s-cni-cncf-io\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.861946    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862041    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-var-lib-cni-multus\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862073    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-system-cni-dir\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862077    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0c1af91b-e37d-4a36-a741-21f2bcb399ac-iptables-alerter-script\") pod \"iptables-alerter-9pczp\" (UID: \"0c1af91b-e37d-4a36-a741-21f2bcb399ac\") " pod="openshift-network-operator/iptables-alerter-9pczp"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862115    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c1af91b-e37d-4a36-a741-21f2bcb399ac-host-slash\") pod \"iptables-alerter-9pczp\" (UID: \"0c1af91b-e37d-4a36-a741-21f2bcb399ac\") " pod="openshift-network-operator/iptables-alerter-9pczp"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862126    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-etc-openvswitch\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862147    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-sysctl-d\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862174    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-run-openvswitch\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.862369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862203    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ffdae73-5e32-418c-8138-171b99a0c118-multus-daemon-config\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862249    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-systemd-units\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862275    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/59d9f229-3224-476f-ae39-e8b4e039bae5-agent-certs\") pod \"konnectivity-agent-pb6hf\" (UID: \"59d9f229-3224-476f-ae39-e8b4e039bae5\") " pod="kube-system/konnectivity-agent-pb6hf"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862303    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f665bfb-5209-4095-9276-31d40e0d326b-cni-binary-copy\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862330    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-var-lib-cni-bin\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862356    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxvfx\" (UniqueName: \"kubernetes.io/projected/1ffdae73-5e32-418c-8138-171b99a0c118-kube-api-access-jxvfx\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862383    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-sysctl-conf\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862407    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f5245560-79ba-41c7-9676-d6278d5b8675-etc-tuned\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862434    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-multus-cni-dir\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862470    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-etc-selinux\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862501    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-modprobe-d\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862530    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-var-lib-openvswitch\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862556    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c01881d-abb4-462f-8807-43242a0e0dbd-ovnkube-config\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862583    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-hostroot\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862611    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqnj5\" (UniqueName: \"kubernetes.io/projected/d23607fd-429c-4a38-acc1-fbac3cd34147-kube-api-access-bqnj5\") pod \"node-resolver-gvwv2\" (UID: \"d23607fd-429c-4a38-acc1-fbac3cd34147\") " pod="openshift-dns/node-resolver-gvwv2"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862631    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0c1af91b-e37d-4a36-a741-21f2bcb399ac-iptables-alerter-script\") pod \"iptables-alerter-9pczp\" (UID: \"0c1af91b-e37d-4a36-a741-21f2bcb399ac\") " pod="openshift-network-operator/iptables-alerter-9pczp"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862636    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-kubelet\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.863291 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862663    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c01881d-abb4-462f-8807-43242a0e0dbd-ovn-node-metrics-cert\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862691    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-socket-dir\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862705    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c1af91b-e37d-4a36-a741-21f2bcb399ac-host-slash\") pod \"iptables-alerter-9pczp\" (UID: \"0c1af91b-e37d-4a36-a741-21f2bcb399ac\") " pod="openshift-network-operator/iptables-alerter-9pczp"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862716    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-sys-fs\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862749    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-sysctl-d\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862766    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-systemd\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862776    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-sys-fs\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862798    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-sys\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862824    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f665bfb-5209-4095-9276-31d40e0d326b-cnibin\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862842    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c01881d-abb4-462f-8807-43242a0e0dbd-ovnkube-script-lib\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862850    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-run-netns\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862874    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-lib-modules\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862897    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-host\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862906    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862920    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-cni-bin\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862935    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d23607fd-429c-4a38-acc1-fbac3cd34147-tmp-dir\") pod \"node-resolver-gvwv2\" (UID: \"d23607fd-429c-4a38-acc1-fbac3cd34147\") " pod="openshift-dns/node-resolver-gvwv2"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862971    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f665bfb-5209-4095-9276-31d40e0d326b-system-cni-dir\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.862996    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-cnibin\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.864125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.863023    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-run-multus-certs\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.863048    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-kubernetes\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.863071    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-node-log\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.863093    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c01881d-abb4-462f-8807-43242a0e0dbd-env-overrides\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.863118    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs\") pod \"network-metrics-daemon-ltd4k\" (UID: \"c907cd72-cf97-4133-87f8-2e9bb7fb67a6\") " pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.863143    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjrkr\" (UniqueName: \"kubernetes.io/projected/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-kube-api-access-cjrkr\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.863164    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-host\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.863168    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g98fg\" (UniqueName: \"kubernetes.io/projected/0c1af91b-e37d-4a36-a741-21f2bcb399ac-kube-api-access-g98fg\") pod \"iptables-alerter-9pczp\" (UID: \"0c1af91b-e37d-4a36-a741-21f2bcb399ac\") " pod="openshift-network-operator/iptables-alerter-9pczp"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.863220    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-log-socket\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.863255    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-registration-dir\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.863285    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-device-dir\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.864007    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-cni-bin\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.864083    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f665bfb-5209-4095-9276-31d40e0d326b-system-cni-dir\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.864150    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-cnibin\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.864208    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-run-multus-certs\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.864276    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-kubernetes\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.864325    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-node-log\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.864762    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c01881d-abb4-462f-8807-43242a0e0dbd-env-overrides\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.864949 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:30.864889    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:52:30.865841 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:30.865015    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs podName:c907cd72-cf97-4133-87f8-2e9bb7fb67a6 nodeName:}" failed. No retries permitted until 2026-09-09 20:52:31.36499327 +0000 UTC m=+3.035622954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs") pod "network-metrics-daemon-ltd4k" (UID: "c907cd72-cf97-4133-87f8-2e9bb7fb67a6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:52:30.865841 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865441    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgbmn\" (UniqueName: \"kubernetes.io/projected/f5245560-79ba-41c7-9676-d6278d5b8675-kube-api-access-lgbmn\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.865841 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865497    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d23607fd-429c-4a38-acc1-fbac3cd34147-hosts-file\") pod \"node-resolver-gvwv2\" (UID: \"d23607fd-429c-4a38-acc1-fbac3cd34147\") " pod="openshift-dns/node-resolver-gvwv2"
Sep 09 20:52:30.865841 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865537    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3f665bfb-5209-4095-9276-31d40e0d326b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.865841 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865574    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-slash\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.865841 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865605    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.865841 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865649    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f665bfb-5209-4095-9276-31d40e0d326b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.865841 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865686    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3f665bfb-5209-4095-9276-31d40e0d326b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.865841 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865707    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f5245560-79ba-41c7-9676-d6278d5b8675-etc-tuned\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.865841 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865720    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-multus-socket-dir-parent\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.865841 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865764    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-run-ovn\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.865841 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865785    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-multus-socket-dir-parent\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.865841 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865793    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-cni-netd\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.865841 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865827    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-cni-netd\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.865841 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865835    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ffdae73-5e32-418c-8138-171b99a0c118-cni-binary-copy\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.866547 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865868    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-run-ovn\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.866547 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865874    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5245560-79ba-41c7-9676-d6278d5b8675-tmp\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.866547 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865916    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-run-netns\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.866547 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865969    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/59d9f229-3224-476f-ae39-e8b4e039bae5-konnectivity-ca\") pod \"konnectivity-agent-pb6hf\" (UID: \"59d9f229-3224-476f-ae39-e8b4e039bae5\") " pod="kube-system/konnectivity-agent-pb6hf"
Sep 09 20:52:30.866547 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.866012    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c74bg\" (UniqueName: \"kubernetes.io/projected/3f665bfb-5209-4095-9276-31d40e0d326b-kube-api-access-c74bg\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.866547 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.866050    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-multus-conf-dir\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.866547 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.866086    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-sysconfig\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.866547 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.866115    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-var-lib-kubelet\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.866547 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.866170    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-var-lib-cni-bin\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.866547 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.866230    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-var-lib-kubelet\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.866547 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.866280    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-slash\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.866547 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.866340    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-etc-selinux\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.866547 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.866353    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d23607fd-429c-4a38-acc1-fbac3cd34147-hosts-file\") pod \"node-resolver-gvwv2\" (UID: \"d23607fd-429c-4a38-acc1-fbac3cd34147\") " pod="openshift-dns/node-resolver-gvwv2"
Sep 09 20:52:30.866547 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.866359    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-multus-cni-dir\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.866547 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.866420    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-var-lib-openvswitch\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.867282 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.865760    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f665bfb-5209-4095-9276-31d40e0d326b-cni-binary-copy\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.867282 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.866648    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-run-netns\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.867282 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.866894    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-kubelet\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.867282 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.866996    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-hostroot\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.867282 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.867034    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c01881d-abb4-462f-8807-43242a0e0dbd-ovnkube-config\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.867282 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.867217    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/59d9f229-3224-476f-ae39-e8b4e039bae5-konnectivity-ca\") pod \"konnectivity-agent-pb6hf\" (UID: \"59d9f229-3224-476f-ae39-e8b4e039bae5\") " pod="kube-system/konnectivity-agent-pb6hf"
Sep 09 20:52:30.867580 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.867393    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-sysctl-conf\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.867580 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.867464    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-systemd\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.867580 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.867529    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-sys\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.867720 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.867576    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-host-run-netns\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.867720 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.867622    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ffdae73-5e32-418c-8138-171b99a0c118-multus-conf-dir\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.867720 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.867635    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-socket-dir\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.867720 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.867684    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-sysconfig\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.867907 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.867729    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f665bfb-5209-4095-9276-31d40e0d326b-cnibin\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.867907 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.867773    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-etc-modprobe-d\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.867907 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.867776    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f5245560-79ba-41c7-9676-d6278d5b8675-lib-modules\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.867907 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.867792    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-registration-dir\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.867907 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.867841    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-systemd-units\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.867907 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.867852    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-log-socket\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.868441 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.868417    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3f665bfb-5209-4095-9276-31d40e0d326b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.868546 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.868423    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3f665bfb-5209-4095-9276-31d40e0d326b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.868546 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.868484    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-device-dir\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.868546 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.868491    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ffdae73-5e32-418c-8138-171b99a0c118-multus-daemon-config\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.868708 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.868559    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ffdae73-5e32-418c-8138-171b99a0c118-cni-binary-copy\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.868708 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.868564    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-run-openvswitch\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.868708 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.868638    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c01881d-abb4-462f-8807-43242a0e0dbd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.868708 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.868692    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f665bfb-5209-4095-9276-31d40e0d326b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.870271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.870157    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/59d9f229-3224-476f-ae39-e8b4e039bae5-agent-certs\") pod \"konnectivity-agent-pb6hf\" (UID: \"59d9f229-3224-476f-ae39-e8b4e039bae5\") " pod="kube-system/konnectivity-agent-pb6hf"
Sep 09 20:52:30.871873 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.871849    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5245560-79ba-41c7-9676-d6278d5b8675-tmp\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.874202 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.874176    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c01881d-abb4-462f-8807-43242a0e0dbd-ovn-node-metrics-cert\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.876679 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.876652    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l9ds\" (UniqueName: \"kubernetes.io/projected/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-kube-api-access-5l9ds\") pod \"network-metrics-daemon-ltd4k\" (UID: \"c907cd72-cf97-4133-87f8-2e9bb7fb67a6\") " pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:30.878088 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:30.878066    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:52:30.878191 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:30.878092    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:52:30.878191 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:30.878107    2585 projected.go:196] Error preparing data for projected volume kube-api-access-m6rp9 for pod openshift-network-diagnostics/network-check-target-4ct29: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:52:30.878191 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:30.878184    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9 podName:04fa9846-4123-477d-8c86-481d721ecc1e nodeName:}" failed. No retries permitted until 2026-09-09 20:52:31.378164178 +0000 UTC m=+3.048793875 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-m6rp9" (UniqueName: "kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9") pod "network-check-target-4ct29" (UID: "04fa9846-4123-477d-8c86-481d721ecc1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:52:30.890258 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.890190    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8bjl\" (UniqueName: \"kubernetes.io/projected/9c01881d-abb4-462f-8807-43242a0e0dbd-kube-api-access-v8bjl\") pod \"ovnkube-node-82b4g\" (UID: \"9c01881d-abb4-462f-8807-43242a0e0dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:30.891228 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.891203    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqnj5\" (UniqueName: \"kubernetes.io/projected/d23607fd-429c-4a38-acc1-fbac3cd34147-kube-api-access-bqnj5\") pod \"node-resolver-gvwv2\" (UID: \"d23607fd-429c-4a38-acc1-fbac3cd34147\") " pod="openshift-dns/node-resolver-gvwv2"
Sep 09 20:52:30.894120 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.893694    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g98fg\" (UniqueName: \"kubernetes.io/projected/0c1af91b-e37d-4a36-a741-21f2bcb399ac-kube-api-access-g98fg\") pod \"iptables-alerter-9pczp\" (UID: \"0c1af91b-e37d-4a36-a741-21f2bcb399ac\") " pod="openshift-network-operator/iptables-alerter-9pczp"
Sep 09 20:52:30.894120 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.894092    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgbmn\" (UniqueName: \"kubernetes.io/projected/f5245560-79ba-41c7-9676-d6278d5b8675-kube-api-access-lgbmn\") pod \"tuned-w5btq\" (UID: \"f5245560-79ba-41c7-9676-d6278d5b8675\") " pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.894811 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.894787    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjrkr\" (UniqueName: \"kubernetes.io/projected/2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b-kube-api-access-cjrkr\") pod \"aws-ebs-csi-driver-node-cxv2q\" (UID: \"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:30.895829 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.895786    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxvfx\" (UniqueName: \"kubernetes.io/projected/1ffdae73-5e32-418c-8138-171b99a0c118-kube-api-access-jxvfx\") pod \"multus-x5c85\" (UID: \"1ffdae73-5e32-418c-8138-171b99a0c118\") " pod="openshift-multus/multus-x5c85"
Sep 09 20:52:30.896032 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.896007    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c74bg\" (UniqueName: \"kubernetes.io/projected/3f665bfb-5209-4095-9276-31d40e0d326b-kube-api-access-c74bg\") pod \"multus-additional-cni-plugins-pwq6l\" (UID: \"3f665bfb-5209-4095-9276-31d40e0d326b\") " pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:30.956247 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.956214    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:52:30.961382 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.961355    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kgvvq"
Sep 09 20:52:30.975776 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.975741    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9pczp"
Sep 09 20:52:30.985356 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.985167    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w5btq"
Sep 09 20:52:30.991451 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.991426    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gvwv2"
Sep 09 20:52:30.999693 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:30.999669    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:31.008128 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:31.008101    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pwq6l"
Sep 09 20:52:31.017226 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:31.017204    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x5c85"
Sep 09 20:52:31.025662 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:31.025639    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pb6hf"
Sep 09 20:52:31.031752 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:31.031729    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q"
Sep 09 20:52:31.268399 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:31.268308    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret\") pod \"global-pull-secret-syncer-c28zc\" (UID: \"95b5dc36-8d5d-43d2-9444-5e870b1ca94b\") " pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:31.268552 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:31.268447    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:52:31.268552 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:31.268521    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret podName:95b5dc36-8d5d-43d2-9444-5e870b1ca94b nodeName:}" failed. No retries permitted until 2026-09-09 20:52:32.268499649 +0000 UTC m=+3.939129335 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret") pod "global-pull-secret-syncer-c28zc" (UID: "95b5dc36-8d5d-43d2-9444-5e870b1ca94b") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:52:31.369609 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:31.369566    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs\") pod \"network-metrics-daemon-ltd4k\" (UID: \"c907cd72-cf97-4133-87f8-2e9bb7fb67a6\") " pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:31.369783 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:31.369732    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:52:31.369870 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:31.369821    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs podName:c907cd72-cf97-4133-87f8-2e9bb7fb67a6 nodeName:}" failed. No retries permitted until 2026-09-09 20:52:32.369797838 +0000 UTC m=+4.040427538 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs") pod "network-metrics-daemon-ltd4k" (UID: "c907cd72-cf97-4133-87f8-2e9bb7fb67a6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:52:31.470204 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:31.470159    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6rp9\" (UniqueName: \"kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9\") pod \"network-check-target-4ct29\" (UID: \"04fa9846-4123-477d-8c86-481d721ecc1e\") " pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:31.470414 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:31.470366    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:52:31.470414 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:31.470391    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:52:31.470414 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:31.470402    2585 projected.go:196] Error preparing data for projected volume kube-api-access-m6rp9 for pod openshift-network-diagnostics/network-check-target-4ct29: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:52:31.470595 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:31.470475    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9 podName:04fa9846-4123-477d-8c86-481d721ecc1e nodeName:}" failed. No retries permitted until 2026-09-09 20:52:32.47044471 +0000 UTC m=+4.141074388 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-m6rp9" (UniqueName: "kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9") pod "network-check-target-4ct29" (UID: "04fa9846-4123-477d-8c86-481d721ecc1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:52:31.710321 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:31.710268    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:47:31 +0000 UTC" deadline="2028-03-17 04:30:09.344858111 +0000 UTC"
Sep 09 20:52:31.710321 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:31.710316    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13303h37m37.634547048s"
Sep 09 20:52:31.793465 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:31.793433    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:31.793650 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:31.793558    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:52:31.904690 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:31.904654    2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:52:32.084242 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:32.084199    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f665bfb_5209_4095_9276_31d40e0d326b.slice/crio-ef9f7e8305536237c76f89a6edbec7ce6612cbee4611e3a260366d5f55d2b4ad WatchSource:0}: Error finding container ef9f7e8305536237c76f89a6edbec7ce6612cbee4611e3a260366d5f55d2b4ad: Status 404 returned error can't find the container with id ef9f7e8305536237c76f89a6edbec7ce6612cbee4611e3a260366d5f55d2b4ad
Sep 09 20:52:32.085664 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:32.085507    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f399b85_c8bd_417f_ae6a_1085b82a4a58.slice/crio-2eea1ce9dcf3ce3e4f4a36d5d39b47f5781e714ee6cbc47eee906bd2aa8a3083 WatchSource:0}: Error finding container 2eea1ce9dcf3ce3e4f4a36d5d39b47f5781e714ee6cbc47eee906bd2aa8a3083: Status 404 returned error can't find the container with id 2eea1ce9dcf3ce3e4f4a36d5d39b47f5781e714ee6cbc47eee906bd2aa8a3083
Sep 09 20:52:32.090157 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:32.090124    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ffdae73_5e32_418c_8138_171b99a0c118.slice/crio-612011497f6b9206656d7033d3d32957f8be08f56c9c04d231b68264e5a3ea98 WatchSource:0}: Error finding container 612011497f6b9206656d7033d3d32957f8be08f56c9c04d231b68264e5a3ea98: Status 404 returned error can't find the container with id 612011497f6b9206656d7033d3d32957f8be08f56c9c04d231b68264e5a3ea98
Sep 09 20:52:32.091593 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:32.091561    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd23607fd_429c_4a38_acc1_fbac3cd34147.slice/crio-d9fd792d8052a615f0f901ce404f0716495f2fdac8027e1db0d664bc76737abd WatchSource:0}: Error finding container d9fd792d8052a615f0f901ce404f0716495f2fdac8027e1db0d664bc76737abd: Status 404 returned error can't find the container with id d9fd792d8052a615f0f901ce404f0716495f2fdac8027e1db0d664bc76737abd
Sep 09 20:52:32.092093 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:32.092068    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d9f229_3224_476f_ae39_e8b4e039bae5.slice/crio-b16809d9d0837ba9edc754e9c7886ac04b62768c4fbe7f2d5706161fcfde53c9 WatchSource:0}: Error finding container b16809d9d0837ba9edc754e9c7886ac04b62768c4fbe7f2d5706161fcfde53c9: Status 404 returned error can't find the container with id b16809d9d0837ba9edc754e9c7886ac04b62768c4fbe7f2d5706161fcfde53c9
Sep 09 20:52:32.093878 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:32.093522    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c01881d_abb4_462f_8807_43242a0e0dbd.slice/crio-c5855b46fbf2301374eaf73fa917c4f722ded724010d66e7d881db81d27460c2 WatchSource:0}: Error finding container c5855b46fbf2301374eaf73fa917c4f722ded724010d66e7d881db81d27460c2: Status 404 returned error can't find the container with id c5855b46fbf2301374eaf73fa917c4f722ded724010d66e7d881db81d27460c2
Sep 09 20:52:32.093878 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:32.093856    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5245560_79ba_41c7_9676_d6278d5b8675.slice/crio-9569e396529a7c51c3250105e93eb33dc48ea72246d64aa1badba6455f1514fb WatchSource:0}: Error finding container 9569e396529a7c51c3250105e93eb33dc48ea72246d64aa1badba6455f1514fb: Status 404 returned error can't find the container with id 9569e396529a7c51c3250105e93eb33dc48ea72246d64aa1badba6455f1514fb
Sep 09 20:52:32.095907 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:52:32.095761    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e7850b5_1a8a_49e1_8d64_9d8e9dd9976b.slice/crio-13400b2c282be625cb4e1112972de90ef1b6d2d953610751ece21b7323834c39 WatchSource:0}: Error finding container 13400b2c282be625cb4e1112972de90ef1b6d2d953610751ece21b7323834c39: Status 404 returned error can't find the container with id 13400b2c282be625cb4e1112972de90ef1b6d2d953610751ece21b7323834c39
Sep 09 20:52:32.276545 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.276506    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret\") pod \"global-pull-secret-syncer-c28zc\" (UID: \"95b5dc36-8d5d-43d2-9444-5e870b1ca94b\") " pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:32.276715 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:32.276692    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:52:32.276772 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:32.276761    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret podName:95b5dc36-8d5d-43d2-9444-5e870b1ca94b nodeName:}" failed. No retries permitted until 2026-09-09 20:52:34.276743896 +0000 UTC m=+5.947373575 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret") pod "global-pull-secret-syncer-c28zc" (UID: "95b5dc36-8d5d-43d2-9444-5e870b1ca94b") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:52:32.377078 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.377043    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs\") pod \"network-metrics-daemon-ltd4k\" (UID: \"c907cd72-cf97-4133-87f8-2e9bb7fb67a6\") " pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:32.377306 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:32.377161    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:52:32.377306 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:32.377221    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs podName:c907cd72-cf97-4133-87f8-2e9bb7fb67a6 nodeName:}" failed. No retries permitted until 2026-09-09 20:52:34.377205139 +0000 UTC m=+6.047834819 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs") pod "network-metrics-daemon-ltd4k" (UID: "c907cd72-cf97-4133-87f8-2e9bb7fb67a6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:52:32.477792 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.477750    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6rp9\" (UniqueName: \"kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9\") pod \"network-check-target-4ct29\" (UID: \"04fa9846-4123-477d-8c86-481d721ecc1e\") " pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:32.477997 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:32.477969    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:52:32.477997 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:32.477996    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:52:32.478135 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:32.478008    2585 projected.go:196] Error preparing data for projected volume kube-api-access-m6rp9 for pod openshift-network-diagnostics/network-check-target-4ct29: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:52:32.478135 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:32.478069    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9 podName:04fa9846-4123-477d-8c86-481d721ecc1e nodeName:}" failed. No retries permitted until 2026-09-09 20:52:34.478053751 +0000 UTC m=+6.148683431 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-m6rp9" (UniqueName: "kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9") pod "network-check-target-4ct29" (UID: "04fa9846-4123-477d-8c86-481d721ecc1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:52:32.711469 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.711339    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:47:31 +0000 UTC" deadline="2028-02-16 19:10:57.895649437 +0000 UTC"
Sep 09 20:52:32.711469 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.711411    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12598h18m25.184243235s"
Sep 09 20:52:32.796282 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.796245    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:32.796452 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:32.796392    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:52:32.796821 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.796799    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:32.796940 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:32.796919    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:52:32.813312 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.813272    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kgvvq" event={"ID":"3f399b85-c8bd-417f-ae6a-1085b82a4a58","Type":"ContainerStarted","Data":"2eea1ce9dcf3ce3e4f4a36d5d39b47f5781e714ee6cbc47eee906bd2aa8a3083"}
Sep 09 20:52:32.817134 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.817102    2585 generic.go:356] "Generic (PLEG): container finished" podID="ee33b1848fc9d62f869c7f00a9a00592" containerID="cd6df0cc2221c7ec46e2d010f509d047d64346550c50e643dc1ff0ac570e47d5" exitCode=0
Sep 09 20:52:32.817463 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.817396    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal" event={"ID":"ee33b1848fc9d62f869c7f00a9a00592","Type":"ContainerDied","Data":"cd6df0cc2221c7ec46e2d010f509d047d64346550c50e643dc1ff0ac570e47d5"}
Sep 09 20:52:32.819783 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.819755    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-24.ec2.internal" event={"ID":"4ad114d159ddf9784199c66d4e552938","Type":"ContainerStarted","Data":"e46b233cbd10875bfc2ceac7c590f4ece633071e30efad4abe1e70909066a889"}
Sep 09 20:52:32.823556 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.823525    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g" event={"ID":"9c01881d-abb4-462f-8807-43242a0e0dbd","Type":"ContainerStarted","Data":"c5855b46fbf2301374eaf73fa917c4f722ded724010d66e7d881db81d27460c2"}
Sep 09 20:52:32.833164 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.833090    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pb6hf" event={"ID":"59d9f229-3224-476f-ae39-e8b4e039bae5","Type":"ContainerStarted","Data":"b16809d9d0837ba9edc754e9c7886ac04b62768c4fbe7f2d5706161fcfde53c9"}
Sep 09 20:52:32.839403 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.839374    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9pczp" event={"ID":"0c1af91b-e37d-4a36-a741-21f2bcb399ac","Type":"ContainerStarted","Data":"5fa30d400151bc818e92d627192ad8958d3ca8f4589399d3a2c6938828836aaf"}
Sep 09 20:52:32.851074 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.851043    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x5c85" event={"ID":"1ffdae73-5e32-418c-8138-171b99a0c118","Type":"ContainerStarted","Data":"612011497f6b9206656d7033d3d32957f8be08f56c9c04d231b68264e5a3ea98"}
Sep 09 20:52:32.851917 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.851861    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-24.ec2.internal" podStartSLOduration=2.851843134 podStartE2EDuration="2.851843134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:52:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:32.851658369 +0000 UTC m=+4.522288075" watchObservedRunningTime="2026-09-09 20:52:32.851843134 +0000 UTC m=+4.522472838"
Sep 09 20:52:32.862892 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.862855    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pwq6l" event={"ID":"3f665bfb-5209-4095-9276-31d40e0d326b","Type":"ContainerStarted","Data":"ef9f7e8305536237c76f89a6edbec7ce6612cbee4611e3a260366d5f55d2b4ad"}
Sep 09 20:52:32.871686 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.871554    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q" event={"ID":"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b","Type":"ContainerStarted","Data":"13400b2c282be625cb4e1112972de90ef1b6d2d953610751ece21b7323834c39"}
Sep 09 20:52:32.876178 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.876148    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w5btq" event={"ID":"f5245560-79ba-41c7-9676-d6278d5b8675","Type":"ContainerStarted","Data":"9569e396529a7c51c3250105e93eb33dc48ea72246d64aa1badba6455f1514fb"}
Sep 09 20:52:32.879379 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:32.879347    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gvwv2" event={"ID":"d23607fd-429c-4a38-acc1-fbac3cd34147","Type":"ContainerStarted","Data":"d9fd792d8052a615f0f901ce404f0716495f2fdac8027e1db0d664bc76737abd"}
Sep 09 20:52:33.795713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:33.793724    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:33.795713 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:33.794227    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:52:33.925678 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:33.925606    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal" event={"ID":"ee33b1848fc9d62f869c7f00a9a00592","Type":"ContainerStarted","Data":"4bf19197d9d2c2407b49326f07a7677ce1e43c76d781aa6e5460a84b25f46640"}
Sep 09 20:52:34.292734 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:34.292078    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret\") pod \"global-pull-secret-syncer-c28zc\" (UID: \"95b5dc36-8d5d-43d2-9444-5e870b1ca94b\") " pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:34.292734 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:34.292244    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:52:34.292734 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:34.292318    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret podName:95b5dc36-8d5d-43d2-9444-5e870b1ca94b nodeName:}" failed. No retries permitted until 2026-09-09 20:52:38.292298191 +0000 UTC m=+9.962927876 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret") pod "global-pull-secret-syncer-c28zc" (UID: "95b5dc36-8d5d-43d2-9444-5e870b1ca94b") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:52:34.392931 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:34.392874    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs\") pod \"network-metrics-daemon-ltd4k\" (UID: \"c907cd72-cf97-4133-87f8-2e9bb7fb67a6\") " pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:34.393170 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:34.393046    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:52:34.393170 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:34.393110    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs podName:c907cd72-cf97-4133-87f8-2e9bb7fb67a6 nodeName:}" failed. No retries permitted until 2026-09-09 20:52:38.393095164 +0000 UTC m=+10.063724843 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs") pod "network-metrics-daemon-ltd4k" (UID: "c907cd72-cf97-4133-87f8-2e9bb7fb67a6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:52:34.494217 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:34.493873    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6rp9\" (UniqueName: \"kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9\") pod \"network-check-target-4ct29\" (UID: \"04fa9846-4123-477d-8c86-481d721ecc1e\") " pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:34.494217 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:34.494135    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:52:34.494217 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:34.494159    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:52:34.494217 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:34.494173    2585 projected.go:196] Error preparing data for projected volume kube-api-access-m6rp9 for pod openshift-network-diagnostics/network-check-target-4ct29: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:52:34.494596 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:34.494239    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9 podName:04fa9846-4123-477d-8c86-481d721ecc1e nodeName:}" failed. No retries permitted until 2026-09-09 20:52:38.494219473 +0000 UTC m=+10.164849158 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-m6rp9" (UniqueName: "kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9") pod "network-check-target-4ct29" (UID: "04fa9846-4123-477d-8c86-481d721ecc1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:52:34.805533 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:34.805492    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:34.806071 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:34.805669    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:52:34.806311 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:34.806285    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:34.806503 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:34.806478    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:52:35.793188 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:35.793149    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:35.793391 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:35.793304    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:52:36.793677 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:36.792964    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:36.793677 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:36.793101    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:52:36.793677 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:36.793500    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:36.793677 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:36.793627    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:52:37.793220 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:37.793178    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:37.793396 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:37.793305    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:52:38.336481 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:38.336442    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret\") pod \"global-pull-secret-syncer-c28zc\" (UID: \"95b5dc36-8d5d-43d2-9444-5e870b1ca94b\") " pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:38.336973 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:38.336703    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:52:38.336973 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:38.336785    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret podName:95b5dc36-8d5d-43d2-9444-5e870b1ca94b nodeName:}" failed. No retries permitted until 2026-09-09 20:52:46.336762287 +0000 UTC m=+18.007391967 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret") pod "global-pull-secret-syncer-c28zc" (UID: "95b5dc36-8d5d-43d2-9444-5e870b1ca94b") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:52:38.437753 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:38.437709    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs\") pod \"network-metrics-daemon-ltd4k\" (UID: \"c907cd72-cf97-4133-87f8-2e9bb7fb67a6\") " pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:38.437984 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:38.437843    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:52:38.437984 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:38.437926    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs podName:c907cd72-cf97-4133-87f8-2e9bb7fb67a6 nodeName:}" failed. No retries permitted until 2026-09-09 20:52:46.437906696 +0000 UTC m=+18.108536376 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs") pod "network-metrics-daemon-ltd4k" (UID: "c907cd72-cf97-4133-87f8-2e9bb7fb67a6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:52:38.538582 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:38.538538    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6rp9\" (UniqueName: \"kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9\") pod \"network-check-target-4ct29\" (UID: \"04fa9846-4123-477d-8c86-481d721ecc1e\") " pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:38.538773 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:38.538699    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:52:38.538773 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:38.538721    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:52:38.538773 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:38.538732    2585 projected.go:196] Error preparing data for projected volume kube-api-access-m6rp9 for pod openshift-network-diagnostics/network-check-target-4ct29: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:52:38.538971 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:38.538791    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9 podName:04fa9846-4123-477d-8c86-481d721ecc1e nodeName:}" failed. No retries permitted until 2026-09-09 20:52:46.538771629 +0000 UTC m=+18.209401322 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-m6rp9" (UniqueName: "kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9") pod "network-check-target-4ct29" (UID: "04fa9846-4123-477d-8c86-481d721ecc1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:52:38.794226 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:38.793735    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:38.794226 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:38.793858    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:52:38.794731 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:38.794569    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:38.794731 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:38.794692    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:52:39.793746 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:39.793302    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:39.793746 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:39.793440    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:52:40.793322 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:40.793282    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:40.793487 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:40.793282    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:40.793487 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:40.793430    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:52:40.793613 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:40.793577    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:52:41.793161 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:41.793132    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:41.793508 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:41.793238    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:52:42.793926 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:42.793890    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:42.794322 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:42.793901    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:42.794322 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:42.794005    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:52:42.794322 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:42.794096    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:52:43.793544 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:43.793500    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:43.793735 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:43.793636    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:52:44.793137 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:44.793105    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:44.793581 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:44.793115    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:44.793581 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:44.793253    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:52:44.793581 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:44.793330    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:52:45.793365 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:45.793326    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:45.793869 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:45.793458    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:52:46.398008 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:46.397967    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret\") pod \"global-pull-secret-syncer-c28zc\" (UID: \"95b5dc36-8d5d-43d2-9444-5e870b1ca94b\") " pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:46.398205 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:46.398105    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:52:46.398205 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:46.398181    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret podName:95b5dc36-8d5d-43d2-9444-5e870b1ca94b nodeName:}" failed. No retries permitted until 2026-09-09 20:53:02.398160503 +0000 UTC m=+34.068790198 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret") pod "global-pull-secret-syncer-c28zc" (UID: "95b5dc36-8d5d-43d2-9444-5e870b1ca94b") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:52:46.498995 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:46.498960    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs\") pod \"network-metrics-daemon-ltd4k\" (UID: \"c907cd72-cf97-4133-87f8-2e9bb7fb67a6\") " pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:46.499170 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:46.499140    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:52:46.499250 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:46.499225    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs podName:c907cd72-cf97-4133-87f8-2e9bb7fb67a6 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:02.499203264 +0000 UTC m=+34.169832963 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs") pod "network-metrics-daemon-ltd4k" (UID: "c907cd72-cf97-4133-87f8-2e9bb7fb67a6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:52:46.599962 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:46.599923    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6rp9\" (UniqueName: \"kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9\") pod \"network-check-target-4ct29\" (UID: \"04fa9846-4123-477d-8c86-481d721ecc1e\") " pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:46.600112 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:46.600070    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:52:46.600112 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:46.600084    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:52:46.600112 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:46.600094    2585 projected.go:196] Error preparing data for projected volume kube-api-access-m6rp9 for pod openshift-network-diagnostics/network-check-target-4ct29: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:52:46.600232 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:46.600145    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9 podName:04fa9846-4123-477d-8c86-481d721ecc1e nodeName:}" failed. No retries permitted until 2026-09-09 20:53:02.600131857 +0000 UTC m=+34.270761535 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-m6rp9" (UniqueName: "kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9") pod "network-check-target-4ct29" (UID: "04fa9846-4123-477d-8c86-481d721ecc1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:52:46.797256 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:46.797226    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:46.797673 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:46.797351    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:52:46.797673 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:46.797411    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:46.797673 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:46.797548    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:52:47.793602 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:47.793571    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:47.793751 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:47.793668    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:52:48.796207 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:48.796176    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:48.796650 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:48.796218    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:48.796650 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:48.796283    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:52:48.796650 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:48.796421    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:52:49.793200 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:49.793158    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:49.793372 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:49.793276    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:52:50.793516 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.793189    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:50.794385 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.793246    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:50.794385 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:50.793626    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:52:50.794385 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:50.793664    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:52:50.971112 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.971081    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x5c85" event={"ID":"1ffdae73-5e32-418c-8138-171b99a0c118","Type":"ContainerStarted","Data":"1c5afe456564ce57dda3dc34515b37ac1b39034aaa9ab5908faa155f49fd16d1"}
Sep 09 20:52:50.972557 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.972527    2585 generic.go:356] "Generic (PLEG): container finished" podID="3f665bfb-5209-4095-9276-31d40e0d326b" containerID="fdc83fd4fcc8e1956a84e762e76b382ac43b7f34903075c92aadd3b44730554e" exitCode=0
Sep 09 20:52:50.972696 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.972588    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pwq6l" event={"ID":"3f665bfb-5209-4095-9276-31d40e0d326b","Type":"ContainerDied","Data":"fdc83fd4fcc8e1956a84e762e76b382ac43b7f34903075c92aadd3b44730554e"}
Sep 09 20:52:50.974191 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.974166    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q" event={"ID":"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b","Type":"ContainerStarted","Data":"800e7b98718873a6aba90a42469291a4b328c81e493a5c2b219806d2c2ddab53"}
Sep 09 20:52:50.975718 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.975701    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w5btq" event={"ID":"f5245560-79ba-41c7-9676-d6278d5b8675","Type":"ContainerStarted","Data":"2245fed0fe6add5ef81712742d8d677fd7b0adec8fc11c9a2e8f473177327e69"}
Sep 09 20:52:50.977135 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.977117    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gvwv2" event={"ID":"d23607fd-429c-4a38-acc1-fbac3cd34147","Type":"ContainerStarted","Data":"babfea59c849824c243cb745bdf38ca52cd27448b765a405314592dde310f9a3"}
Sep 09 20:52:50.978485 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.978465    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kgvvq" event={"ID":"3f399b85-c8bd-417f-ae6a-1085b82a4a58","Type":"ContainerStarted","Data":"50dcf1631cfdd0ec59369e0b534387bf8ad94dcd418398f85663b4e808cf3151"}
Sep 09 20:52:50.981831 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.981812    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g" event={"ID":"9c01881d-abb4-462f-8807-43242a0e0dbd","Type":"ContainerStarted","Data":"9933bf1be30bfa331537ded8f5508a25858ea93065e56f2223cbedeb1ede5510"}
Sep 09 20:52:50.981931 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.981898    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g" event={"ID":"9c01881d-abb4-462f-8807-43242a0e0dbd","Type":"ContainerStarted","Data":"07a1aa7e755070342bb2d19378a5b1e4c7024f911b434845364a629bb62ef16a"}
Sep 09 20:52:50.981979 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.981946    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g" event={"ID":"9c01881d-abb4-462f-8807-43242a0e0dbd","Type":"ContainerStarted","Data":"5fd2facc53a63b475121428ffc5498e0463c180313309d4954dd62c8fc9bec86"}
Sep 09 20:52:50.982019 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.981987    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g" event={"ID":"9c01881d-abb4-462f-8807-43242a0e0dbd","Type":"ContainerStarted","Data":"d799f2b87180d8a83c708631bc9fcd40a0ea30166bdff814eafbfb281933f6f7"}
Sep 09 20:52:50.982059 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.982027    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g" event={"ID":"9c01881d-abb4-462f-8807-43242a0e0dbd","Type":"ContainerStarted","Data":"b7406be0da58adf8cf83c4c025a00d1238459232ced148a0d96c3fe38ad41197"}
Sep 09 20:52:50.982059 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.982056    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g" event={"ID":"9c01881d-abb4-462f-8807-43242a0e0dbd","Type":"ContainerStarted","Data":"814bd519e6a30789e4fa7968fedec0dd3dba1a164dad3237961448e0b635f6fe"}
Sep 09 20:52:50.983160 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.983143    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pb6hf" event={"ID":"59d9f229-3224-476f-ae39-e8b4e039bae5","Type":"ContainerStarted","Data":"ef2ee9fc4f8bb43e96075b6810b89b00275338a293b6e188ab753ff115efec6a"}
Sep 09 20:52:50.993114 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.993075    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-24.ec2.internal" podStartSLOduration=20.993063269 podStartE2EDuration="20.993063269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:52:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:33.95656383 +0000 UTC m=+5.627193537" watchObservedRunningTime="2026-09-09 20:52:50.993063269 +0000 UTC m=+22.663692968"
Sep 09 20:52:50.993200 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:50.993140    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-x5c85" podStartSLOduration=3.970425264 podStartE2EDuration="21.993135081s" totalImagesPullingTime="18.022709817s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:50.993041418 +0000 UTC m=+22.663671119" watchObservedRunningTime="2026-09-09 20:52:50.993135081 +0000 UTC m=+22.663764789"
Sep 09 20:52:51.012309 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:51.012264    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gvwv2" podStartSLOduration=4.056801609 podStartE2EDuration="22.012251091s" totalImagesPullingTime="17.955449482s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:51.011917075 +0000 UTC m=+22.682546776" watchObservedRunningTime="2026-09-09 20:52:51.012251091 +0000 UTC m=+22.682880790"
Sep 09 20:52:51.034470 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:51.034427    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pb6hf" podStartSLOduration=7.596495779 podStartE2EDuration="22.034412766s" totalImagesPullingTime="14.437916987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:51.034261438 +0000 UTC m=+22.704891138" watchObservedRunningTime="2026-09-09 20:52:51.034412766 +0000 UTC m=+22.705042465"
Sep 09 20:52:51.059443 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:51.059378    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kgvvq" podStartSLOduration=4.0983994 podStartE2EDuration="22.05935734s" totalImagesPullingTime="17.96095794s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:51.059032843 +0000 UTC m=+22.729662543" watchObservedRunningTime="2026-09-09 20:52:51.05935734 +0000 UTC m=+22.729987043"
Sep 09 20:52:51.147113 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:51.147013    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-w5btq" podStartSLOduration=4.193384353 podStartE2EDuration="22.146995837s" totalImagesPullingTime="17.953611484s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:51.114757709 +0000 UTC m=+22.785387413" watchObservedRunningTime="2026-09-09 20:52:51.146995837 +0000 UTC m=+22.817625537"
Sep 09 20:52:51.543237 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:51.543141    2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 20:52:51.643971 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:51.643932    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pb6hf"
Sep 09 20:52:51.644509 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:51.644487    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pb6hf"
Sep 09 20:52:51.713897 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:51.713723    2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T20:52:51.543160745Z","UUID":"0eac036f-55cb-48c0-942f-eb79712b8f2e","Handler":null,"Name":"","Endpoint":""}
Sep 09 20:52:51.715471 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:51.715446    2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 20:52:51.715598 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:51.715484    2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 20:52:51.793398 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:51.793367    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:51.793565 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:51.793483    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:52:51.987766 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:51.987691    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q" event={"ID":"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b","Type":"ContainerStarted","Data":"b5acfb8361acf96fe180195535d41ab51a661143f91e7799bb4233bc8c75ef6a"}
Sep 09 20:52:51.989458 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:51.989432    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9pczp" event={"ID":"0c1af91b-e37d-4a36-a741-21f2bcb399ac","Type":"ContainerStarted","Data":"d8006ea6bde1e64ac78065e02477a3cb58559a8a8559f827891ce2158477590f"}
Sep 09 20:52:51.990453 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:51.990433    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pb6hf"
Sep 09 20:52:51.990894 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:51.990863    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pb6hf"
Sep 09 20:52:52.009008 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:52.008961    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9pczp" podStartSLOduration=5.060337206 podStartE2EDuration="23.008944809s" totalImagesPullingTime="17.948607603s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:52.008612305 +0000 UTC m=+23.679242004" watchObservedRunningTime="2026-09-09 20:52:52.008944809 +0000 UTC m=+23.679574509"
Sep 09 20:52:52.793627 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:52.793601    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:52.794028 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:52.793734    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:52:52.794151 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:52.794132    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:52.794274 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:52.794250    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:52:52.996315 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:52.996227    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g" event={"ID":"9c01881d-abb4-462f-8807-43242a0e0dbd","Type":"ContainerStarted","Data":"e80791f842a02a679b2c3b4066e60b73e87fd131e38312c77912aa2737f6fdfa"}
Sep 09 20:52:52.998745 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:52.998678    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q" event={"ID":"2e7850b5-1a8a-49e1-8d64-9d8e9dd9976b","Type":"ContainerStarted","Data":"b391071781f74a15ab1353b834a30fca7c44bce3afa00d888f7f58d545ce94c8"}
Sep 09 20:52:53.021362 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:53.021304    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cxv2q" podStartSLOduration=3.679524037 podStartE2EDuration="24.021288591s" totalImagesPullingTime="20.341764554s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:52:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:53.020963643 +0000 UTC m=+24.691593344" watchObservedRunningTime="2026-09-09 20:52:53.021288591 +0000 UTC m=+24.691918292"
Sep 09 20:52:53.793767 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:53.793730    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:53.794219 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:53.793855    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:52:54.797552 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:54.797524    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:54.797978 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:54.797625    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:52:54.797978 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:54.797678    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:54.797978 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:54.797785    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:52:55.793468 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:55.793292    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:55.793633 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:55.793552    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:52:56.009083 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:56.009048    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g" event={"ID":"9c01881d-abb4-462f-8807-43242a0e0dbd","Type":"ContainerStarted","Data":"543cfbce44df86acec036719ffadb224da27a8efad5004bd29ca6833d0aad882"}
Sep 09 20:52:56.009977 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:56.009286    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:56.009977 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:56.009358    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:56.011062 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:56.011038    2585 generic.go:356] "Generic (PLEG): container finished" podID="3f665bfb-5209-4095-9276-31d40e0d326b" containerID="201988b855d4a2a136e42e0b9f82b30b2470827fc60b1ce07886a3d154251d08" exitCode=0
Sep 09 20:52:56.011162 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:56.011081    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pwq6l" event={"ID":"3f665bfb-5209-4095-9276-31d40e0d326b","Type":"ContainerDied","Data":"201988b855d4a2a136e42e0b9f82b30b2470827fc60b1ce07886a3d154251d08"}
Sep 09 20:52:56.024942 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:56.024919    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:56.053985 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:56.053907    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g" podStartSLOduration=9.040447204 podStartE2EDuration="27.053874316s" totalImagesPullingTime="18.013427112s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:56.053675498 +0000 UTC m=+27.724305199" watchObservedRunningTime="2026-09-09 20:52:56.053874316 +0000 UTC m=+27.724504016"
Sep 09 20:52:56.797376 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:56.797346    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:56.797541 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:56.797467    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:52:56.797541 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:56.797500    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:56.797652 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:56.797564    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:52:57.014670 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:57.014164    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:57.029454 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:57.029430    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:52:57.123417 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:57.123318    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c28zc"]
Sep 09 20:52:57.123417 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:57.123410    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:57.123604 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:57.123503    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:52:57.127161 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:57.127137    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4ct29"]
Sep 09 20:52:57.127282 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:57.127232    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:57.127343 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:57.127312    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:52:57.131264 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:57.131242    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ltd4k"]
Sep 09 20:52:57.131369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:57.131350    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:57.131461 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:57.131440    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:52:58.018361 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:58.018327    2585 generic.go:356] "Generic (PLEG): container finished" podID="3f665bfb-5209-4095-9276-31d40e0d326b" containerID="7b112775c5faf399b628724da63404d69d46a9267cf1c01de9ed2f29d7ab5ac7" exitCode=0
Sep 09 20:52:58.018714 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:58.018422    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pwq6l" event={"ID":"3f665bfb-5209-4095-9276-31d40e0d326b","Type":"ContainerDied","Data":"7b112775c5faf399b628724da63404d69d46a9267cf1c01de9ed2f29d7ab5ac7"}
Sep 09 20:52:58.794455 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:58.794429    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:52:58.794580 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:58.794543    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:52:58.794679 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:58.794539    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:52:58.794679 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:58.794659    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:52:58.794798 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:58.794687    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:52:58.794856 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:52:58.794802    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:52:59.023639 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:59.023609    2585 generic.go:356] "Generic (PLEG): container finished" podID="3f665bfb-5209-4095-9276-31d40e0d326b" containerID="c4d285643732a11893bf9191686c4f147782ea6db1b3a4623007d59c041f73b2" exitCode=0
Sep 09 20:52:59.023980 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:52:59.023710    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pwq6l" event={"ID":"3f665bfb-5209-4095-9276-31d40e0d326b","Type":"ContainerDied","Data":"c4d285643732a11893bf9191686c4f147782ea6db1b3a4623007d59c041f73b2"}
Sep 09 20:53:00.797138 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:00.797109    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:53:00.797138 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:00.797136    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:53:00.797591 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:00.797120    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:53:00.797591 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:00.797246    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:53:00.797591 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:00.797381    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:53:00.797591 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:00.797467    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:53:02.414253 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:02.414212    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret\") pod \"global-pull-secret-syncer-c28zc\" (UID: \"95b5dc36-8d5d-43d2-9444-5e870b1ca94b\") " pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:53:02.414779 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:02.414372    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:53:02.414779 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:02.414449    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret podName:95b5dc36-8d5d-43d2-9444-5e870b1ca94b nodeName:}" failed. No retries permitted until 2026-09-09 20:53:34.414431934 +0000 UTC m=+66.085061616 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret") pod "global-pull-secret-syncer-c28zc" (UID: "95b5dc36-8d5d-43d2-9444-5e870b1ca94b") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:53:02.514930 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:02.514889    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs\") pod \"network-metrics-daemon-ltd4k\" (UID: \"c907cd72-cf97-4133-87f8-2e9bb7fb67a6\") " pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:53:02.515096 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:02.515040    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:53:02.515202 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:02.515124    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs podName:c907cd72-cf97-4133-87f8-2e9bb7fb67a6 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:34.515103689 +0000 UTC m=+66.185733369 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs") pod "network-metrics-daemon-ltd4k" (UID: "c907cd72-cf97-4133-87f8-2e9bb7fb67a6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:53:02.616310 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:02.616275    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6rp9\" (UniqueName: \"kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9\") pod \"network-check-target-4ct29\" (UID: \"04fa9846-4123-477d-8c86-481d721ecc1e\") " pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:53:02.616492 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:02.616425    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:53:02.616492 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:02.616455    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:53:02.616492 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:02.616467    2585 projected.go:196] Error preparing data for projected volume kube-api-access-m6rp9 for pod openshift-network-diagnostics/network-check-target-4ct29: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:53:02.616649 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:02.616540    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9 podName:04fa9846-4123-477d-8c86-481d721ecc1e nodeName:}" failed. No retries permitted until 2026-09-09 20:53:34.616518637 +0000 UTC m=+66.287148339 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-m6rp9" (UniqueName: "kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9") pod "network-check-target-4ct29" (UID: "04fa9846-4123-477d-8c86-481d721ecc1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:53:02.793269 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:02.793234    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:53:02.793452 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:02.793234    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:53:02.793452 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:02.793342    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c28zc" podUID="95b5dc36-8d5d-43d2-9444-5e870b1ca94b"
Sep 09 20:53:02.793452 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:02.793368    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:53:02.793452 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:02.793443    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4ct29" podUID="04fa9846-4123-477d-8c86-481d721ecc1e"
Sep 09 20:53:02.793666 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:02.793499    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ltd4k" podUID="c907cd72-cf97-4133-87f8-2e9bb7fb67a6"
Sep 09 20:53:04.038032 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.037756    2585 generic.go:356] "Generic (PLEG): container finished" podID="3f665bfb-5209-4095-9276-31d40e0d326b" containerID="edf2ee6db2fe1cddfbad7c744840dcd57197a8a23f6eacd13d1258153b50df65" exitCode=0
Sep 09 20:53:04.038432 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.037820    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pwq6l" event={"ID":"3f665bfb-5209-4095-9276-31d40e0d326b","Type":"ContainerDied","Data":"edf2ee6db2fe1cddfbad7c744840dcd57197a8a23f6eacd13d1258153b50df65"}
Sep 09 20:53:04.149928 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.149904    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-24.ec2.internal" event="NodeReady"
Sep 09 20:53:04.150048 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.150019    2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 20:53:04.205711 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.205679    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d55d49d5-29jms"]
Sep 09 20:53:04.224825 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.224799    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6bcgn"]
Sep 09 20:53:04.224966 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.224929    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.228061 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.228029    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 20:53:04.228061 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.228042    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 20:53:04.228245 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.228066    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 20:53:04.231144 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.231120    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bdddp\""
Sep 09 20:53:04.234544 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.234527    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 20:53:04.241031 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.240969    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gc9kl"]
Sep 09 20:53:04.241119 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.241047    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6bcgn"
Sep 09 20:53:04.244867 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.244848    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 20:53:04.245480 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.245450    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 20:53:04.245480 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.245463    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 20:53:04.245480 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.245450    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rcsnd\""
Sep 09 20:53:04.255131 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.255108    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d55d49d5-29jms"]
Sep 09 20:53:04.255244 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.255140    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gc9kl"]
Sep 09 20:53:04.255244 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.255164    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6bcgn"]
Sep 09 20:53:04.255318 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.255246    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:04.258311 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.258290    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 20:53:04.258419 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.258338    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 20:53:04.258739 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.258724    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6lpmt\""
Sep 09 20:53:04.328338 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.328313    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6d02d8e-f8f3-4ade-9074-abd603c3e688-installation-pull-secrets\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.328480 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.328346    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6d02d8e-f8f3-4ade-9074-abd603c3e688-trusted-ca\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.328480 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.328370    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mw2g\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-kube-api-access-7mw2g\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.328480 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.328422    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6d02d8e-f8f3-4ade-9074-abd603c3e688-image-registry-private-configuration\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.328480 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.328448    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:04.328480 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.328477    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e74c0d43-20df-4f97-9b0f-654dd80de9c3-tmp-dir\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:04.328648 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.328495    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnpzw\" (UniqueName: \"kubernetes.io/projected/e74c0d43-20df-4f97-9b0f-654dd80de9c3-kube-api-access-bnpzw\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:04.328648 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.328511    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19c41e9b-65f1-4408-be09-392b760bd6a5-cert\") pod \"ingress-canary-6bcgn\" (UID: \"19c41e9b-65f1-4408-be09-392b760bd6a5\") " pod="openshift-ingress-canary/ingress-canary-6bcgn"
Sep 09 20:53:04.328648 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.328578    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.328648 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.328603    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-certificates\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.328648 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.328637    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e74c0d43-20df-4f97-9b0f-654dd80de9c3-config-volume\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:04.328791 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.328653    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2mml\" (UniqueName: \"kubernetes.io/projected/19c41e9b-65f1-4408-be09-392b760bd6a5-kube-api-access-c2mml\") pod \"ingress-canary-6bcgn\" (UID: \"19c41e9b-65f1-4408-be09-392b760bd6a5\") " pod="openshift-ingress-canary/ingress-canary-6bcgn"
Sep 09 20:53:04.328791 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.328670    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6d02d8e-f8f3-4ade-9074-abd603c3e688-ca-trust-extracted\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.328791 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.328694    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-bound-sa-token\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.429685 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.429649    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-bound-sa-token\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.429860 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.429721    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6d02d8e-f8f3-4ade-9074-abd603c3e688-installation-pull-secrets\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.429860 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.429750    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6d02d8e-f8f3-4ade-9074-abd603c3e688-trusted-ca\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.429860 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.429772    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mw2g\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-kube-api-access-7mw2g\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.429860 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.429798    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6d02d8e-f8f3-4ade-9074-abd603c3e688-image-registry-private-configuration\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.430131 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.429941    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:04.430131 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.429981    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e74c0d43-20df-4f97-9b0f-654dd80de9c3-tmp-dir\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:04.430131 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.430009    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnpzw\" (UniqueName: \"kubernetes.io/projected/e74c0d43-20df-4f97-9b0f-654dd80de9c3-kube-api-access-bnpzw\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:04.430131 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:04.430019    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:53:04.430131 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.430037    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19c41e9b-65f1-4408-be09-392b760bd6a5-cert\") pod \"ingress-canary-6bcgn\" (UID: \"19c41e9b-65f1-4408-be09-392b760bd6a5\") " pod="openshift-ingress-canary/ingress-canary-6bcgn"
Sep 09 20:53:04.430131 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:04.430095    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls podName:e74c0d43-20df-4f97-9b0f-654dd80de9c3 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:04.930074572 +0000 UTC m=+36.600704253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls") pod "dns-default-gc9kl" (UID: "e74c0d43-20df-4f97-9b0f-654dd80de9c3") : secret "dns-default-metrics-tls" not found
Sep 09 20:53:04.430428 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.430138    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.430428 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.430182    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-certificates\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.430428 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.430211    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e74c0d43-20df-4f97-9b0f-654dd80de9c3-config-volume\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:04.430428 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.430235    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2mml\" (UniqueName: \"kubernetes.io/projected/19c41e9b-65f1-4408-be09-392b760bd6a5-kube-api-access-c2mml\") pod \"ingress-canary-6bcgn\" (UID: \"19c41e9b-65f1-4408-be09-392b760bd6a5\") " pod="openshift-ingress-canary/ingress-canary-6bcgn"
Sep 09 20:53:04.430428 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.430270    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6d02d8e-f8f3-4ade-9074-abd603c3e688-ca-trust-extracted\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.430428 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.430293    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e74c0d43-20df-4f97-9b0f-654dd80de9c3-tmp-dir\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:04.430428 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:04.430388    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:53:04.430428 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:04.430403    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d55d49d5-29jms: secret "image-registry-tls" not found
Sep 09 20:53:04.430787 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:04.430444    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls podName:a6d02d8e-f8f3-4ade-9074-abd603c3e688 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:04.930428169 +0000 UTC m=+36.601057860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls") pod "image-registry-7d55d49d5-29jms" (UID: "a6d02d8e-f8f3-4ade-9074-abd603c3e688") : secret "image-registry-tls" not found
Sep 09 20:53:04.430787 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.430717    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-certificates\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.430969 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.430861    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6d02d8e-f8f3-4ade-9074-abd603c3e688-ca-trust-extracted\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.430969 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.430959    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6d02d8e-f8f3-4ade-9074-abd603c3e688-trusted-ca\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.431109 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.431086    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e74c0d43-20df-4f97-9b0f-654dd80de9c3-config-volume\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:04.433931 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.433906    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6d02d8e-f8f3-4ade-9074-abd603c3e688-image-registry-private-configuration\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.433931 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.433913    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6d02d8e-f8f3-4ade-9074-abd603c3e688-installation-pull-secrets\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.434063 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.434034    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19c41e9b-65f1-4408-be09-392b760bd6a5-cert\") pod \"ingress-canary-6bcgn\" (UID: \"19c41e9b-65f1-4408-be09-392b760bd6a5\") " pod="openshift-ingress-canary/ingress-canary-6bcgn"
Sep 09 20:53:04.439963 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.439944    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-bound-sa-token\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.443313 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.443291    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnpzw\" (UniqueName: \"kubernetes.io/projected/e74c0d43-20df-4f97-9b0f-654dd80de9c3-kube-api-access-bnpzw\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:04.443407 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.443298    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2mml\" (UniqueName: \"kubernetes.io/projected/19c41e9b-65f1-4408-be09-392b760bd6a5-kube-api-access-c2mml\") pod \"ingress-canary-6bcgn\" (UID: \"19c41e9b-65f1-4408-be09-392b760bd6a5\") " pod="openshift-ingress-canary/ingress-canary-6bcgn"
Sep 09 20:53:04.444096 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.444076    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mw2g\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-kube-api-access-7mw2g\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.598633 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.598543    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6bcgn"
Sep 09 20:53:04.793901 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.793846    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:53:04.793901 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.793896    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:53:04.794157 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.794139    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:53:04.798618 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.798435    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 20:53:04.798618 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.798437    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 20:53:04.798618 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.798488    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 20:53:04.798618 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.798495    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 20:53:04.798618 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.798534    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-thpvv\""
Sep 09 20:53:04.798931 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.798913    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pstzn\""
Sep 09 20:53:04.925987 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.925909    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6bcgn"]
Sep 09 20:53:04.933486 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:53:04.933451    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19c41e9b_65f1_4408_be09_392b760bd6a5.slice/crio-6e54f9e53dea4d59c25b47ec4028ccbf6d172ccda927c2f0e0f61c1c17066a24 WatchSource:0}: Error finding container 6e54f9e53dea4d59c25b47ec4028ccbf6d172ccda927c2f0e0f61c1c17066a24: Status 404 returned error can't find the container with id 6e54f9e53dea4d59c25b47ec4028ccbf6d172ccda927c2f0e0f61c1c17066a24
Sep 09 20:53:04.933693 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.933673    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:04.933769 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:04.933736    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:04.933868 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:04.933856    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:53:04.933933 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:04.933870    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d55d49d5-29jms: secret "image-registry-tls" not found
Sep 09 20:53:04.934042 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:04.933944    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls podName:a6d02d8e-f8f3-4ade-9074-abd603c3e688 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:05.933928578 +0000 UTC m=+37.604558256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls") pod "image-registry-7d55d49d5-29jms" (UID: "a6d02d8e-f8f3-4ade-9074-abd603c3e688") : secret "image-registry-tls" not found
Sep 09 20:53:04.934042 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:04.933966    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:53:04.934042 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:04.934031    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls podName:e74c0d43-20df-4f97-9b0f-654dd80de9c3 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:05.934018008 +0000 UTC m=+37.604647690 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls") pod "dns-default-gc9kl" (UID: "e74c0d43-20df-4f97-9b0f-654dd80de9c3") : secret "dns-default-metrics-tls" not found
Sep 09 20:53:05.043569 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:05.043531    2585 generic.go:356] "Generic (PLEG): container finished" podID="3f665bfb-5209-4095-9276-31d40e0d326b" containerID="5e31f06a96ef0af3423ba9c0e014ab7c22c17d528bfe1df8b1ddffa49bb044a1" exitCode=0
Sep 09 20:53:05.044315 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:05.043608    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pwq6l" event={"ID":"3f665bfb-5209-4095-9276-31d40e0d326b","Type":"ContainerDied","Data":"5e31f06a96ef0af3423ba9c0e014ab7c22c17d528bfe1df8b1ddffa49bb044a1"}
Sep 09 20:53:05.044902 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:05.044865    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6bcgn" event={"ID":"19c41e9b-65f1-4408-be09-392b760bd6a5","Type":"ContainerStarted","Data":"6e54f9e53dea4d59c25b47ec4028ccbf6d172ccda927c2f0e0f61c1c17066a24"}
Sep 09 20:53:05.944566 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:05.944370    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:05.944757 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:05.944621    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:05.944757 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:05.944532    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:53:05.944757 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:05.944656    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d55d49d5-29jms: secret "image-registry-tls" not found
Sep 09 20:53:05.944757 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:05.944730    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:53:05.944757 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:05.944734    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls podName:a6d02d8e-f8f3-4ade-9074-abd603c3e688 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:07.944715478 +0000 UTC m=+39.615345173 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls") pod "image-registry-7d55d49d5-29jms" (UID: "a6d02d8e-f8f3-4ade-9074-abd603c3e688") : secret "image-registry-tls" not found
Sep 09 20:53:05.944935 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:05.944774    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls podName:e74c0d43-20df-4f97-9b0f-654dd80de9c3 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:07.944763859 +0000 UTC m=+39.615393540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls") pod "dns-default-gc9kl" (UID: "e74c0d43-20df-4f97-9b0f-654dd80de9c3") : secret "dns-default-metrics-tls" not found
Sep 09 20:53:06.050485 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.050455    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pwq6l" event={"ID":"3f665bfb-5209-4095-9276-31d40e0d326b","Type":"ContainerStarted","Data":"52cea189d8949de9547865248f8ab85b2832d756a8343e278a2b73f87066d63d"}
Sep 09 20:53:06.077205 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.077160    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pwq6l" podStartSLOduration=8.383001675 podStartE2EDuration="37.077144618s" totalImagesPullingTime="28.694142943s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:52:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:06.075841 +0000 UTC m=+37.746470701" watchObservedRunningTime="2026-09-09 20:53:06.077144618 +0000 UTC m=+37.747774314"
Sep 09 20:53:06.564298 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.564266    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kzn2d"]
Sep 09 20:53:06.570318 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.570289    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:06.578271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.578067    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 20:53:06.578271 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.578070    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 20:53:06.578951 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.578935    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 20:53:06.579524 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.579504    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fptv8\""
Sep 09 20:53:06.579614 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.579534    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 20:53:06.592749 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.592719    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kzn2d"]
Sep 09 20:53:06.652021 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.651842    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/befc00a5-d23a-41b4-b476-af36f0ed0cd9-crio-socket\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:06.652021 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.652006    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:06.652265 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.652068    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/befc00a5-d23a-41b4-b476-af36f0ed0cd9-data-volume\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:06.652265 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.652100    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/befc00a5-d23a-41b4-b476-af36f0ed0cd9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:06.652265 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.652172    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnx55\" (UniqueName: \"kubernetes.io/projected/befc00a5-d23a-41b4-b476-af36f0ed0cd9-kube-api-access-bnx55\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:06.685492 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.685457    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vk9h4"]
Sep 09 20:53:06.706369 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.706337    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vk9h4"]
Sep 09 20:53:06.706649 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.706470    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vk9h4"
Sep 09 20:53:06.709615 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.709590    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 09 20:53:06.709777 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.709595    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-75x86\""
Sep 09 20:53:06.710389 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.710370    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 09 20:53:06.752579 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.752549    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnx55\" (UniqueName: \"kubernetes.io/projected/befc00a5-d23a-41b4-b476-af36f0ed0cd9-kube-api-access-bnx55\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:06.752751 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.752640    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/befc00a5-d23a-41b4-b476-af36f0ed0cd9-crio-socket\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:06.752751 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.752660    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:06.752751 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.752681    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/befc00a5-d23a-41b4-b476-af36f0ed0cd9-data-volume\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:06.752751 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.752700    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/befc00a5-d23a-41b4-b476-af36f0ed0cd9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:06.752992 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:06.752835    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:06.752992 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:06.752937    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls podName:befc00a5-d23a-41b4-b476-af36f0ed0cd9 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:07.252915706 +0000 UTC m=+38.923545389 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kzn2d" (UID: "befc00a5-d23a-41b4-b476-af36f0ed0cd9") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:06.752992 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:06.752939    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:53:06.752992 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:06.752981    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/befc00a5-d23a-41b4-b476-af36f0ed0cd9-kube-rbac-proxy-cm podName:befc00a5-d23a-41b4-b476-af36f0ed0cd9 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:07.252974177 +0000 UTC m=+38.923603855 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/befc00a5-d23a-41b4-b476-af36f0ed0cd9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kzn2d" (UID: "befc00a5-d23a-41b4-b476-af36f0ed0cd9") : configmap "kube-rbac-proxy" not found
Sep 09 20:53:06.752992 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.752987    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/befc00a5-d23a-41b4-b476-af36f0ed0cd9-crio-socket\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:06.753178 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.753069    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/befc00a5-d23a-41b4-b476-af36f0ed0cd9-data-volume\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:06.766553 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.766523    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnx55\" (UniqueName: \"kubernetes.io/projected/befc00a5-d23a-41b4-b476-af36f0ed0cd9-kube-api-access-bnx55\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:06.853450 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.853369    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx69x\" (UniqueName: \"kubernetes.io/projected/adb45051-79cf-498b-81b4-5f483067b937-kube-api-access-nx69x\") pod \"migrator-fdbb8fdff-vk9h4\" (UID: \"adb45051-79cf-498b-81b4-5f483067b937\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vk9h4"
Sep 09 20:53:06.954286 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.954243    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx69x\" (UniqueName: \"kubernetes.io/projected/adb45051-79cf-498b-81b4-5f483067b937-kube-api-access-nx69x\") pod \"migrator-fdbb8fdff-vk9h4\" (UID: \"adb45051-79cf-498b-81b4-5f483067b937\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vk9h4"
Sep 09 20:53:06.968138 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:06.968111    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx69x\" (UniqueName: \"kubernetes.io/projected/adb45051-79cf-498b-81b4-5f483067b937-kube-api-access-nx69x\") pod \"migrator-fdbb8fdff-vk9h4\" (UID: \"adb45051-79cf-498b-81b4-5f483067b937\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vk9h4"
Sep 09 20:53:07.018360 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:07.018328    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vk9h4"
Sep 09 20:53:07.256669 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:07.256579    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:07.256669 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:07.256637    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/befc00a5-d23a-41b4-b476-af36f0ed0cd9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:07.257178 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:07.256710    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:53:07.257178 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:07.256752    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:07.257178 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:07.256787    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/befc00a5-d23a-41b4-b476-af36f0ed0cd9-kube-rbac-proxy-cm podName:befc00a5-d23a-41b4-b476-af36f0ed0cd9 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:08.2567724 +0000 UTC m=+39.927402078 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/befc00a5-d23a-41b4-b476-af36f0ed0cd9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kzn2d" (UID: "befc00a5-d23a-41b4-b476-af36f0ed0cd9") : configmap "kube-rbac-proxy" not found
Sep 09 20:53:07.257178 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:07.256829    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls podName:befc00a5-d23a-41b4-b476-af36f0ed0cd9 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:08.256810123 +0000 UTC m=+39.927439807 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kzn2d" (UID: "befc00a5-d23a-41b4-b476-af36f0ed0cd9") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:07.305098 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:07.305068    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vk9h4"]
Sep 09 20:53:07.842740 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:53:07.842710    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadb45051_79cf_498b_81b4_5f483067b937.slice/crio-17ae9948a949ff4aab71c48be06a24cdfac8e2148d30b12d2dfe1f63050badee WatchSource:0}: Error finding container 17ae9948a949ff4aab71c48be06a24cdfac8e2148d30b12d2dfe1f63050badee: Status 404 returned error can't find the container with id 17ae9948a949ff4aab71c48be06a24cdfac8e2148d30b12d2dfe1f63050badee
Sep 09 20:53:07.916008 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:07.915831    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gvwv2_d23607fd-429c-4a38-acc1-fbac3cd34147/dns-node-resolver/0.log"
Sep 09 20:53:07.963172 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:07.963133    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:07.963336 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:07.963226    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:07.963336 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:07.963286    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:53:07.963336 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:07.963304    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d55d49d5-29jms: secret "image-registry-tls" not found
Sep 09 20:53:07.963336 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:07.963330    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:53:07.963481 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:07.963364    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls podName:a6d02d8e-f8f3-4ade-9074-abd603c3e688 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:11.96334584 +0000 UTC m=+43.633975519 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls") pod "image-registry-7d55d49d5-29jms" (UID: "a6d02d8e-f8f3-4ade-9074-abd603c3e688") : secret "image-registry-tls" not found
Sep 09 20:53:07.963481 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:07.963381    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls podName:e74c0d43-20df-4f97-9b0f-654dd80de9c3 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:11.963369312 +0000 UTC m=+43.633999013 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls") pod "dns-default-gc9kl" (UID: "e74c0d43-20df-4f97-9b0f-654dd80de9c3") : secret "dns-default-metrics-tls" not found
Sep 09 20:53:08.057245 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:08.057215    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vk9h4" event={"ID":"adb45051-79cf-498b-81b4-5f483067b937","Type":"ContainerStarted","Data":"17ae9948a949ff4aab71c48be06a24cdfac8e2148d30b12d2dfe1f63050badee"}
Sep 09 20:53:08.266031 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:08.266003    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/befc00a5-d23a-41b4-b476-af36f0ed0cd9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:08.266511 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:08.266159    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:53:08.266511 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:08.266184    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:08.266511 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:08.266228    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/befc00a5-d23a-41b4-b476-af36f0ed0cd9-kube-rbac-proxy-cm podName:befc00a5-d23a-41b4-b476-af36f0ed0cd9 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:10.266209517 +0000 UTC m=+41.936839217 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/befc00a5-d23a-41b4-b476-af36f0ed0cd9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kzn2d" (UID: "befc00a5-d23a-41b4-b476-af36f0ed0cd9") : configmap "kube-rbac-proxy" not found
Sep 09 20:53:08.266511 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:08.266294    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:08.266511 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:08.266340    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls podName:befc00a5-d23a-41b4-b476-af36f0ed0cd9 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:10.266328614 +0000 UTC m=+41.936958315 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kzn2d" (UID: "befc00a5-d23a-41b4-b476-af36f0ed0cd9") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:09.061774 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:09.061727    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6bcgn" event={"ID":"19c41e9b-65f1-4408-be09-392b760bd6a5","Type":"ContainerStarted","Data":"28003a7ecdb9fc5a8645f9fa1e1624a612290f729a27535d83a104c078076615"}
Sep 09 20:53:09.082560 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:09.082505    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6bcgn" podStartSLOduration=1.833339327 podStartE2EDuration="5.08248864s" totalImagesPullingTime="3.249149313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:09.08104337 +0000 UTC m=+40.751673070" watchObservedRunningTime="2026-09-09 20:53:09.08248864 +0000 UTC m=+40.753118340"
Sep 09 20:53:09.317914 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:09.317821    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kgvvq_3f399b85-c8bd-417f-ae6a-1085b82a4a58/node-ca/0.log"
Sep 09 20:53:09.717690 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:09.717656    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6bcgn_19c41e9b-65f1-4408-be09-392b760bd6a5/serve-healthcheck-canary/0.log"
Sep 09 20:53:10.066019 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:10.065979    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vk9h4" event={"ID":"adb45051-79cf-498b-81b4-5f483067b937","Type":"ContainerStarted","Data":"5fef4cbca4879bcffc077e360ebd18c3e3017b29d7e4a8e3a046bbbbfa296155"}
Sep 09 20:53:10.066215 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:10.066070    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vk9h4" event={"ID":"adb45051-79cf-498b-81b4-5f483067b937","Type":"ContainerStarted","Data":"acd669268bc95ac3050778c011ac7dca5bb4509daf2de1e3a895ef8fb4d4bd04"}
Sep 09 20:53:10.088580 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:10.088532    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-vk9h4" podStartSLOduration=2.59299491 podStartE2EDuration="4.088515853s" totalImagesPullingTime="1.495520943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:10.088079626 +0000 UTC m=+41.758709325" watchObservedRunningTime="2026-09-09 20:53:10.088515853 +0000 UTC m=+41.759145571"
Sep 09 20:53:10.282613 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:10.282575    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:10.282613 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:10.282621    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/befc00a5-d23a-41b4-b476-af36f0ed0cd9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:10.282872 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:10.282734    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:10.282872 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:10.282831    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls podName:befc00a5-d23a-41b4-b476-af36f0ed0cd9 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:14.28280972 +0000 UTC m=+45.953439405 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kzn2d" (UID: "befc00a5-d23a-41b4-b476-af36f0ed0cd9") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:10.283198 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:10.283177    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/befc00a5-d23a-41b4-b476-af36f0ed0cd9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:11.994315 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:11.994272    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:11.994755 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:11.994341    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:11.994755 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:11.994436    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:53:11.994755 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:11.994447    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:53:11.994755 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:11.994474    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d55d49d5-29jms: secret "image-registry-tls" not found
Sep 09 20:53:11.994755 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:11.994489    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls podName:e74c0d43-20df-4f97-9b0f-654dd80de9c3 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:19.994476489 +0000 UTC m=+51.665106167 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls") pod "dns-default-gc9kl" (UID: "e74c0d43-20df-4f97-9b0f-654dd80de9c3") : secret "dns-default-metrics-tls" not found
Sep 09 20:53:11.994755 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:11.994527    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls podName:a6d02d8e-f8f3-4ade-9074-abd603c3e688 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:19.99451089 +0000 UTC m=+51.665140568 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls") pod "image-registry-7d55d49d5-29jms" (UID: "a6d02d8e-f8f3-4ade-9074-abd603c3e688") : secret "image-registry-tls" not found
Sep 09 20:53:14.311587 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:14.311543    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:14.312082 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:14.311681    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:14.312082 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:14.311760    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls podName:befc00a5-d23a-41b4-b476-af36f0ed0cd9 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:22.311739243 +0000 UTC m=+53.982368924 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kzn2d" (UID: "befc00a5-d23a-41b4-b476-af36f0ed0cd9") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:20.058630 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:20.058591    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:20.059102 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:20.058653    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:20.061744 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:20.061721    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e74c0d43-20df-4f97-9b0f-654dd80de9c3-metrics-tls\") pod \"dns-default-gc9kl\" (UID: \"e74c0d43-20df-4f97-9b0f-654dd80de9c3\") " pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:20.061822 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:20.061769    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls\") pod \"image-registry-7d55d49d5-29jms\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") " pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:20.192894 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:20.192858    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:20.203742 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:20.203709    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:20.499129 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:20.499035    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d55d49d5-29jms"]
Sep 09 20:53:20.500053 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:20.500028    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gc9kl"]
Sep 09 20:53:20.501066 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:53:20.501042    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode74c0d43_20df_4f97_9b0f_654dd80de9c3.slice/crio-5a8d94a7e66fe22abf766eb2085b1cc8740b4447f08f4e38c66d732c06c9ac1c WatchSource:0}: Error finding container 5a8d94a7e66fe22abf766eb2085b1cc8740b4447f08f4e38c66d732c06c9ac1c: Status 404 returned error can't find the container with id 5a8d94a7e66fe22abf766eb2085b1cc8740b4447f08f4e38c66d732c06c9ac1c
Sep 09 20:53:20.501630 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:53:20.501601    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6d02d8e_f8f3_4ade_9074_abd603c3e688.slice/crio-1cf4dca04a1e3d0ac220666aea22b50da79fca33699434212a0cd8dafb9db132 WatchSource:0}: Error finding container 1cf4dca04a1e3d0ac220666aea22b50da79fca33699434212a0cd8dafb9db132: Status 404 returned error can't find the container with id 1cf4dca04a1e3d0ac220666aea22b50da79fca33699434212a0cd8dafb9db132
Sep 09 20:53:21.093221 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:21.093187    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gc9kl" event={"ID":"e74c0d43-20df-4f97-9b0f-654dd80de9c3","Type":"ContainerStarted","Data":"5a8d94a7e66fe22abf766eb2085b1cc8740b4447f08f4e38c66d732c06c9ac1c"}
Sep 09 20:53:21.094841 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:21.094816    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d55d49d5-29jms" event={"ID":"a6d02d8e-f8f3-4ade-9074-abd603c3e688","Type":"ContainerStarted","Data":"699cd3477c435ecadbbef94b4e5cdbd29a52f2be1886c5b99de8f62b960f4b72"}
Sep 09 20:53:21.094995 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:21.094857    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d55d49d5-29jms" event={"ID":"a6d02d8e-f8f3-4ade-9074-abd603c3e688","Type":"ContainerStarted","Data":"1cf4dca04a1e3d0ac220666aea22b50da79fca33699434212a0cd8dafb9db132"}
Sep 09 20:53:21.094995 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:21.094973    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:21.122598 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:21.122547    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d55d49d5-29jms" podStartSLOduration=24.122533818 podStartE2EDuration="24.122533818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:52:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:21.120954156 +0000 UTC m=+52.791583856" watchObservedRunningTime="2026-09-09 20:53:21.122533818 +0000 UTC m=+52.793163517"
Sep 09 20:53:22.099201 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:22.099175    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gc9kl" event={"ID":"e74c0d43-20df-4f97-9b0f-654dd80de9c3","Type":"ContainerStarted","Data":"5a0df307d8fdd31b6b78847ac89b51b4859984ef26416f0ce2d935abe76e01ba"}
Sep 09 20:53:22.376897 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:22.376851    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:22.379134 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:22.379103    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/befc00a5-d23a-41b4-b476-af36f0ed0cd9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kzn2d\" (UID: \"befc00a5-d23a-41b4-b476-af36f0ed0cd9\") " pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:22.481450 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:22.481414    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kzn2d"
Sep 09 20:53:22.753000 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:22.752915    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kzn2d"]
Sep 09 20:53:22.756010 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:53:22.755977    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbefc00a5_d23a_41b4_b476_af36f0ed0cd9.slice/crio-194e14f891d6c3ab2db44567ab764246808ef43e0686d77f6fa04bcec21e207d WatchSource:0}: Error finding container 194e14f891d6c3ab2db44567ab764246808ef43e0686d77f6fa04bcec21e207d: Status 404 returned error can't find the container with id 194e14f891d6c3ab2db44567ab764246808ef43e0686d77f6fa04bcec21e207d
Sep 09 20:53:23.104112 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:23.104080    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kzn2d" event={"ID":"befc00a5-d23a-41b4-b476-af36f0ed0cd9","Type":"ContainerStarted","Data":"8819ea16c3cd5f8adee39f31e4825d0286fdf175f3b14ad414312a71b39002a6"}
Sep 09 20:53:23.104558 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:23.104140    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kzn2d" event={"ID":"befc00a5-d23a-41b4-b476-af36f0ed0cd9","Type":"ContainerStarted","Data":"194e14f891d6c3ab2db44567ab764246808ef43e0686d77f6fa04bcec21e207d"}
Sep 09 20:53:23.105866 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:23.105844    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gc9kl" event={"ID":"e74c0d43-20df-4f97-9b0f-654dd80de9c3","Type":"ContainerStarted","Data":"188d6dafa9a6de4958c7253c41b00991f0c4c6e4868ae595ee0d6e8aee78f7b7"}
Sep 09 20:53:23.106004 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:23.105989    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:23.127366 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:23.127322    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gc9kl" podStartSLOduration=17.616599642 podStartE2EDuration="19.127303449s" totalImagesPullingTime="1.510703807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:23.125532161 +0000 UTC m=+54.796161861" watchObservedRunningTime="2026-09-09 20:53:23.127303449 +0000 UTC m=+54.797933149"
Sep 09 20:53:24.114304 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:24.114269    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kzn2d" event={"ID":"befc00a5-d23a-41b4-b476-af36f0ed0cd9","Type":"ContainerStarted","Data":"e3a83564a219da7a35739b8d5363a2ba2d5efaf9f57b09047c0a449133603c11"}
Sep 09 20:53:25.119063 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:25.119025    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kzn2d" event={"ID":"befc00a5-d23a-41b4-b476-af36f0ed0cd9","Type":"ContainerStarted","Data":"386021772bebc490b92c9f964b2d93ef9ca8922b9822bade4bb2ade23f63e8b4"}
Sep 09 20:53:25.157275 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:25.157215    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kzn2d" podStartSLOduration=17.537180483 podStartE2EDuration="19.15719799s" totalImagesPullingTime="1.620017507s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:53:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:25.156376559 +0000 UTC m=+56.827006272" watchObservedRunningTime="2026-09-09 20:53:25.15719799 +0000 UTC m=+56.827827740"
Sep 09 20:53:29.034632 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:29.034602    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-82b4g"
Sep 09 20:53:31.346716 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.346684    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf"]
Sep 09 20:53:31.350003 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.349985    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf"
Sep 09 20:53:31.354474 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.354453    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 09 20:53:31.355602 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:31.355572    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"work-manager-hub-kubeconfig\" is forbidden: User \"system:node:ip-10-0-129-24.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-129-24.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" type="*v1.Secret"
Sep 09 20:53:31.356136 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.355819    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 09 20:53:31.356136 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.355832    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 09 20:53:31.378425 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.378387    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c75cb4666-qgrn5"]
Sep 09 20:53:31.380273 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.380252    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c75cb4666-qgrn5"
Sep 09 20:53:31.389475 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.389447    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf"]
Sep 09 20:53:31.390479 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.390450    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d55d49d5-29jms"]
Sep 09 20:53:31.429787 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.429746    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 09 20:53:31.436469 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.436447    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-7wgwp\""
Sep 09 20:53:31.438532 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.438514    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqctr\" (UniqueName: \"kubernetes.io/projected/b8588ef5-4c79-47fb-9655-114f3c172380-kube-api-access-gqctr\") pod \"klusterlet-addon-workmgr-5854878559-6thtf\" (UID: \"b8588ef5-4c79-47fb-9655-114f3c172380\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf"
Sep 09 20:53:31.438587 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.438544    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b8588ef5-4c79-47fb-9655-114f3c172380-klusterlet-config\") pod \"klusterlet-addon-workmgr-5854878559-6thtf\" (UID: \"b8588ef5-4c79-47fb-9655-114f3c172380\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf"
Sep 09 20:53:31.438587 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.438563    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d6ce8d6-2de8-4e9d-8b15-f0c7cef46ba2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-c75cb4666-qgrn5\" (UID: \"2d6ce8d6-2de8-4e9d-8b15-f0c7cef46ba2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c75cb4666-qgrn5"
Sep 09 20:53:31.438587 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.438582    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8588ef5-4c79-47fb-9655-114f3c172380-tmp\") pod \"klusterlet-addon-workmgr-5854878559-6thtf\" (UID: \"b8588ef5-4c79-47fb-9655-114f3c172380\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf"
Sep 09 20:53:31.438723 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.438699    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95x9s\" (UniqueName: \"kubernetes.io/projected/2d6ce8d6-2de8-4e9d-8b15-f0c7cef46ba2-kube-api-access-95x9s\") pod \"managed-serviceaccount-addon-agent-c75cb4666-qgrn5\" (UID: \"2d6ce8d6-2de8-4e9d-8b15-f0c7cef46ba2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c75cb4666-qgrn5"
Sep 09 20:53:31.447569 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.447541    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bf8bb684-lx2vs"]
Sep 09 20:53:31.449548 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.449530    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.454987 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.454957    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 09 20:53:31.454987 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.454970    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 09 20:53:31.455218 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.454987    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 09 20:53:31.455218 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.455069    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 09 20:53:31.455377 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.455357    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kgh55\""
Sep 09 20:53:31.455613 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.455365    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 09 20:53:31.455613 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.455408    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 09 20:53:31.455613 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.455455    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 09 20:53:31.471242 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.471218    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c75cb4666-qgrn5"]
Sep 09 20:53:31.483894 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.483848    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bf8bb684-lx2vs"]
Sep 09 20:53:31.540014 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.539978    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8588ef5-4c79-47fb-9655-114f3c172380-tmp\") pod \"klusterlet-addon-workmgr-5854878559-6thtf\" (UID: \"b8588ef5-4c79-47fb-9655-114f3c172380\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf"
Sep 09 20:53:31.540208 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.540040    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b8588ef5-4c79-47fb-9655-114f3c172380-klusterlet-config\") pod \"klusterlet-addon-workmgr-5854878559-6thtf\" (UID: \"b8588ef5-4c79-47fb-9655-114f3c172380\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf"
Sep 09 20:53:31.540208 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.540072    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4zfh\" (UniqueName: \"kubernetes.io/projected/2724eaa6-cd7a-4ede-9d66-49417b3fb015-kube-api-access-d4zfh\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.540208 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.540115    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-serving-cert\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.540208 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.540140    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqctr\" (UniqueName: \"kubernetes.io/projected/b8588ef5-4c79-47fb-9655-114f3c172380-kube-api-access-gqctr\") pod \"klusterlet-addon-workmgr-5854878559-6thtf\" (UID: \"b8588ef5-4c79-47fb-9655-114f3c172380\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf"
Sep 09 20:53:31.540208 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.540165    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d6ce8d6-2de8-4e9d-8b15-f0c7cef46ba2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-c75cb4666-qgrn5\" (UID: \"2d6ce8d6-2de8-4e9d-8b15-f0c7cef46ba2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c75cb4666-qgrn5"
Sep 09 20:53:31.540208 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.540203    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95x9s\" (UniqueName: \"kubernetes.io/projected/2d6ce8d6-2de8-4e9d-8b15-f0c7cef46ba2-kube-api-access-95x9s\") pod \"managed-serviceaccount-addon-agent-c75cb4666-qgrn5\" (UID: \"2d6ce8d6-2de8-4e9d-8b15-f0c7cef46ba2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c75cb4666-qgrn5"
Sep 09 20:53:31.540495 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.540225    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-oauth-config\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.540495 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.540253    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-config\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.540495 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.540277    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-oauth-serving-cert\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.540495 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.540320    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-service-ca\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.540636 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.540505    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8588ef5-4c79-47fb-9655-114f3c172380-tmp\") pod \"klusterlet-addon-workmgr-5854878559-6thtf\" (UID: \"b8588ef5-4c79-47fb-9655-114f3c172380\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf"
Sep 09 20:53:31.542826 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.542807    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d6ce8d6-2de8-4e9d-8b15-f0c7cef46ba2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-c75cb4666-qgrn5\" (UID: \"2d6ce8d6-2de8-4e9d-8b15-f0c7cef46ba2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c75cb4666-qgrn5"
Sep 09 20:53:31.569896 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.569852    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d84dd5b4f-mjmz8"]
Sep 09 20:53:31.570811 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.570787    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqctr\" (UniqueName: \"kubernetes.io/projected/b8588ef5-4c79-47fb-9655-114f3c172380-kube-api-access-gqctr\") pod \"klusterlet-addon-workmgr-5854878559-6thtf\" (UID: \"b8588ef5-4c79-47fb-9655-114f3c172380\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf"
Sep 09 20:53:31.572092 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.572078    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.587806 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.587777    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95x9s\" (UniqueName: \"kubernetes.io/projected/2d6ce8d6-2de8-4e9d-8b15-f0c7cef46ba2-kube-api-access-95x9s\") pod \"managed-serviceaccount-addon-agent-c75cb4666-qgrn5\" (UID: \"2d6ce8d6-2de8-4e9d-8b15-f0c7cef46ba2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c75cb4666-qgrn5"
Sep 09 20:53:31.612294 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.612219    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d84dd5b4f-mjmz8"]
Sep 09 20:53:31.640853 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.640811    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a82a19d7-0128-4d70-a667-370ad882f303-registry-tls\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.641036 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.640874    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4zfh\" (UniqueName: \"kubernetes.io/projected/2724eaa6-cd7a-4ede-9d66-49417b3fb015-kube-api-access-d4zfh\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.641036 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.640971    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a82a19d7-0128-4d70-a667-370ad882f303-bound-sa-token\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.641036 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.641016    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4fwm\" (UniqueName: \"kubernetes.io/projected/a82a19d7-0128-4d70-a667-370ad882f303-kube-api-access-f4fwm\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.641145 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.641045    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a82a19d7-0128-4d70-a667-370ad882f303-trusted-ca\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.641145 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.641069    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a82a19d7-0128-4d70-a667-370ad882f303-installation-pull-secrets\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.641145 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.641125    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-serving-cert\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.641234 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.641188    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a82a19d7-0128-4d70-a667-370ad882f303-registry-certificates\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.641234 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.641207    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a82a19d7-0128-4d70-a667-370ad882f303-ca-trust-extracted\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.641296 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.641265    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-oauth-config\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.641296 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.641286    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-config\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.641360 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.641310    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-oauth-serving-cert\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.641360 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.641327    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a82a19d7-0128-4d70-a667-370ad882f303-image-registry-private-configuration\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.641433 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.641363    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-service-ca\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.642115 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.642093    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-config\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.642115 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.642095    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-oauth-serving-cert\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.642303 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.642097    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-service-ca\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.643487 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.643466    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-oauth-config\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.643570 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.643549    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-serving-cert\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.667375 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.667344    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4zfh\" (UniqueName: \"kubernetes.io/projected/2724eaa6-cd7a-4ede-9d66-49417b3fb015-kube-api-access-d4zfh\") pod \"console-6bf8bb684-lx2vs\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") " pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.699595 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.699552    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c75cb4666-qgrn5"
Sep 09 20:53:31.742268 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.742219    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a82a19d7-0128-4d70-a667-370ad882f303-image-registry-private-configuration\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.742446 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.742397    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a82a19d7-0128-4d70-a667-370ad882f303-registry-tls\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.742497 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.742487    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a82a19d7-0128-4d70-a667-370ad882f303-bound-sa-token\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.742541 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.742511    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4fwm\" (UniqueName: \"kubernetes.io/projected/a82a19d7-0128-4d70-a667-370ad882f303-kube-api-access-f4fwm\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.742541 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.742533    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a82a19d7-0128-4d70-a667-370ad882f303-trusted-ca\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.742636 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.742549    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a82a19d7-0128-4d70-a667-370ad882f303-installation-pull-secrets\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.742636 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.742590    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a82a19d7-0128-4d70-a667-370ad882f303-registry-certificates\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.742812 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.742782    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a82a19d7-0128-4d70-a667-370ad882f303-ca-trust-extracted\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.743300 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.743260    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a82a19d7-0128-4d70-a667-370ad882f303-ca-trust-extracted\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.743497 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.743452    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a82a19d7-0128-4d70-a667-370ad882f303-registry-certificates\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.744572 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.744501    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a82a19d7-0128-4d70-a667-370ad882f303-trusted-ca\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.746317 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.745475    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a82a19d7-0128-4d70-a667-370ad882f303-registry-tls\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.746317 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.745724    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a82a19d7-0128-4d70-a667-370ad882f303-image-registry-private-configuration\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.746922 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.746899    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a82a19d7-0128-4d70-a667-370ad882f303-installation-pull-secrets\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.755496 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.755466    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a82a19d7-0128-4d70-a667-370ad882f303-bound-sa-token\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.755962 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.755916    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4fwm\" (UniqueName: \"kubernetes.io/projected/a82a19d7-0128-4d70-a667-370ad882f303-kube-api-access-f4fwm\") pod \"image-registry-d84dd5b4f-mjmz8\" (UID: \"a82a19d7-0128-4d70-a667-370ad882f303\") " pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.758716 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.758696    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:31.883329 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.883231    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:31.980819 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:31.980786    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c75cb4666-qgrn5"]
Sep 09 20:53:31.983658 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:53:31.983631    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d6ce8d6_2de8_4e9d_8b15_f0c7cef46ba2.slice/crio-7d4efefc538c3ef9dc5a1a828babca9bc40d31b769d9f2e67f72cca8f94de59a WatchSource:0}: Error finding container 7d4efefc538c3ef9dc5a1a828babca9bc40d31b769d9f2e67f72cca8f94de59a: Status 404 returned error can't find the container with id 7d4efefc538c3ef9dc5a1a828babca9bc40d31b769d9f2e67f72cca8f94de59a
Sep 09 20:53:32.041362 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:32.041323    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bf8bb684-lx2vs"]
Sep 09 20:53:32.046868 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:53:32.046841    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2724eaa6_cd7a_4ede_9d66_49417b3fb015.slice/crio-22404cf3a11c2bcffb3cb87f53de39110df12326646b7972bd8fcc6831f9bd50 WatchSource:0}: Error finding container 22404cf3a11c2bcffb3cb87f53de39110df12326646b7972bd8fcc6831f9bd50: Status 404 returned error can't find the container with id 22404cf3a11c2bcffb3cb87f53de39110df12326646b7972bd8fcc6831f9bd50
Sep 09 20:53:32.140211 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:32.140131    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c75cb4666-qgrn5" event={"ID":"2d6ce8d6-2de8-4e9d-8b15-f0c7cef46ba2","Type":"ContainerStarted","Data":"7d4efefc538c3ef9dc5a1a828babca9bc40d31b769d9f2e67f72cca8f94de59a"}
Sep 09 20:53:32.141405 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:32.141379    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf8bb684-lx2vs" event={"ID":"2724eaa6-cd7a-4ede-9d66-49417b3fb015","Type":"ContainerStarted","Data":"22404cf3a11c2bcffb3cb87f53de39110df12326646b7972bd8fcc6831f9bd50"}
Sep 09 20:53:32.159664 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:32.159643    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d84dd5b4f-mjmz8"]
Sep 09 20:53:32.161756 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:53:32.161734    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda82a19d7_0128_4d70_a667_370ad882f303.slice/crio-d13aaa1e0da607dec97c891f34157928375e344bd36b2d957bc992191a238d81 WatchSource:0}: Error finding container d13aaa1e0da607dec97c891f34157928375e344bd36b2d957bc992191a238d81: Status 404 returned error can't find the container with id d13aaa1e0da607dec97c891f34157928375e344bd36b2d957bc992191a238d81
Sep 09 20:53:32.218045 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:32.218018    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 09 20:53:32.223062 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:32.223043    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b8588ef5-4c79-47fb-9655-114f3c172380-klusterlet-config\") pod \"klusterlet-addon-workmgr-5854878559-6thtf\" (UID: \"b8588ef5-4c79-47fb-9655-114f3c172380\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf"
Sep 09 20:53:32.258767 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:32.258729    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf"
Sep 09 20:53:32.568537 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:32.568507    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf"]
Sep 09 20:53:32.571947 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:53:32.571903    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8588ef5_4c79_47fb_9655_114f3c172380.slice/crio-5c0126e7d4588c9007867814aaddb46e26feb0aaebe57f5b9ea16fa7ca9c82ae WatchSource:0}: Error finding container 5c0126e7d4588c9007867814aaddb46e26feb0aaebe57f5b9ea16fa7ca9c82ae: Status 404 returned error can't find the container with id 5c0126e7d4588c9007867814aaddb46e26feb0aaebe57f5b9ea16fa7ca9c82ae
Sep 09 20:53:33.117276 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:33.117243    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gc9kl"
Sep 09 20:53:33.152803 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:33.152012    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf" event={"ID":"b8588ef5-4c79-47fb-9655-114f3c172380","Type":"ContainerStarted","Data":"5c0126e7d4588c9007867814aaddb46e26feb0aaebe57f5b9ea16fa7ca9c82ae"}
Sep 09 20:53:33.158063 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:33.158027    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8" event={"ID":"a82a19d7-0128-4d70-a667-370ad882f303","Type":"ContainerStarted","Data":"59484d2c253b9476e5688d43b1e69efc8b1891300677e2b6d19f54b55801a098"}
Sep 09 20:53:33.158211 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:33.158094    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8" event={"ID":"a82a19d7-0128-4d70-a667-370ad882f303","Type":"ContainerStarted","Data":"d13aaa1e0da607dec97c891f34157928375e344bd36b2d957bc992191a238d81"}
Sep 09 20:53:33.158211 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:33.158143    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:33.210516 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:33.210270    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8" podStartSLOduration=2.21024688 podStartE2EDuration="2.21024688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:33.208640676 +0000 UTC m=+64.879270377" watchObservedRunningTime="2026-09-09 20:53:33.21024688 +0000 UTC m=+64.880876574"
Sep 09 20:53:34.467683 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:34.467633    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret\") pod \"global-pull-secret-syncer-c28zc\" (UID: \"95b5dc36-8d5d-43d2-9444-5e870b1ca94b\") " pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:53:34.471540 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:34.471303    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 20:53:34.481211 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:34.481181    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95b5dc36-8d5d-43d2-9444-5e870b1ca94b-original-pull-secret\") pod \"global-pull-secret-syncer-c28zc\" (UID: \"95b5dc36-8d5d-43d2-9444-5e870b1ca94b\") " pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:53:34.516535 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:34.516497    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c28zc"
Sep 09 20:53:34.568600 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:34.568557    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs\") pod \"network-metrics-daemon-ltd4k\" (UID: \"c907cd72-cf97-4133-87f8-2e9bb7fb67a6\") " pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:53:34.571937 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:34.571910    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 20:53:34.582563 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:34.582513    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c907cd72-cf97-4133-87f8-2e9bb7fb67a6-metrics-certs\") pod \"network-metrics-daemon-ltd4k\" (UID: \"c907cd72-cf97-4133-87f8-2e9bb7fb67a6\") " pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:53:34.669327 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:34.669291    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6rp9\" (UniqueName: \"kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9\") pod \"network-check-target-4ct29\" (UID: \"04fa9846-4123-477d-8c86-481d721ecc1e\") " pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:53:34.672675 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:34.672651    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 20:53:34.683795 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:34.683770    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 20:53:34.694025 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:34.693991    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6rp9\" (UniqueName: \"kubernetes.io/projected/04fa9846-4123-477d-8c86-481d721ecc1e-kube-api-access-m6rp9\") pod \"network-check-target-4ct29\" (UID: \"04fa9846-4123-477d-8c86-481d721ecc1e\") " pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:53:34.812751 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:34.812716    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-thpvv\""
Sep 09 20:53:34.814673 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:34.814648    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pstzn\""
Sep 09 20:53:34.816781 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:34.816642    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:53:34.821775 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:34.821754    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ltd4k"
Sep 09 20:53:38.181783 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:38.181498    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c75cb4666-qgrn5" event={"ID":"2d6ce8d6-2de8-4e9d-8b15-f0c7cef46ba2","Type":"ContainerStarted","Data":"e5c9a8a9ca477865cb70b1dfd72064ccee9572047353e85d369a55b6bbaf37ea"}
Sep 09 20:53:38.188784 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:38.188751    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf" event={"ID":"b8588ef5-4c79-47fb-9655-114f3c172380","Type":"ContainerStarted","Data":"4192b9b9fff652bae4f3561664ec7af60362fa918390c2a5ee988aeead0b66b0"}
Sep 09 20:53:38.189174 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:38.189127    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf"
Sep 09 20:53:38.191035 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:38.191011    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf8bb684-lx2vs" event={"ID":"2724eaa6-cd7a-4ede-9d66-49417b3fb015","Type":"ContainerStarted","Data":"194f3cf9aebedb6893ba5d6b782f62514f9e0574b814bed34610c824be975375"}
Sep 09 20:53:38.192114 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:38.192095    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf"
Sep 09 20:53:38.202656 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:38.202576    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-c75cb4666-qgrn5" podStartSLOduration=1.149107401 podStartE2EDuration="7.202558567s" totalImagesPullingTime="6.053451166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:38.200989844 +0000 UTC m=+69.871619550" watchObservedRunningTime="2026-09-09 20:53:38.202558567 +0000 UTC m=+69.873188269"
Sep 09 20:53:38.220447 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:38.220371    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bf8bb684-lx2vs" podStartSLOduration=1.252655748 podStartE2EDuration="7.220347328s" totalImagesPullingTime="5.96769158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:38.219644405 +0000 UTC m=+69.890274105" watchObservedRunningTime="2026-09-09 20:53:38.220347328 +0000 UTC m=+69.890977073"
Sep 09 20:53:38.244681 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:38.244624    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5854878559-6thtf" podStartSLOduration=1.75568263 podStartE2EDuration="7.244607492s" totalImagesPullingTime="5.488924862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:38.243588636 +0000 UTC m=+69.914218332" watchObservedRunningTime="2026-09-09 20:53:38.244607492 +0000 UTC m=+69.915237191"
Sep 09 20:53:38.294462 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:38.294437    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c28zc"]
Sep 09 20:53:38.296727 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:53:38.296700    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b5dc36_8d5d_43d2_9444_5e870b1ca94b.slice/crio-861245e17cdab4980dcadc3cc4bf6c0bca58e8e5c61148617396481d11df8547 WatchSource:0}: Error finding container 861245e17cdab4980dcadc3cc4bf6c0bca58e8e5c61148617396481d11df8547: Status 404 returned error can't find the container with id 861245e17cdab4980dcadc3cc4bf6c0bca58e8e5c61148617396481d11df8547
Sep 09 20:53:38.331512 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:38.331480    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4ct29"]
Sep 09 20:53:38.334696 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:53:38.334661    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04fa9846_4123_477d_8c86_481d721ecc1e.slice/crio-313dc82b7076d58dfd9daedb8d3cc495d91f309ebd1c70b9c24372d4d782dc24 WatchSource:0}: Error finding container 313dc82b7076d58dfd9daedb8d3cc495d91f309ebd1c70b9c24372d4d782dc24: Status 404 returned error can't find the container with id 313dc82b7076d58dfd9daedb8d3cc495d91f309ebd1c70b9c24372d4d782dc24
Sep 09 20:53:38.339492 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:38.339462    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ltd4k"]
Sep 09 20:53:38.342730 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:53:38.342706    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc907cd72_cf97_4133_87f8_2e9bb7fb67a6.slice/crio-920fe3994d13984c77c941e69668eba383d7cfa21ca05585585b62090fc7e4c0 WatchSource:0}: Error finding container 920fe3994d13984c77c941e69668eba383d7cfa21ca05585585b62090fc7e4c0: Status 404 returned error can't find the container with id 920fe3994d13984c77c941e69668eba383d7cfa21ca05585585b62090fc7e4c0
Sep 09 20:53:39.198187 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:39.198142    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4ct29" event={"ID":"04fa9846-4123-477d-8c86-481d721ecc1e","Type":"ContainerStarted","Data":"313dc82b7076d58dfd9daedb8d3cc495d91f309ebd1c70b9c24372d4d782dc24"}
Sep 09 20:53:39.199939 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:39.199905    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c28zc" event={"ID":"95b5dc36-8d5d-43d2-9444-5e870b1ca94b","Type":"ContainerStarted","Data":"861245e17cdab4980dcadc3cc4bf6c0bca58e8e5c61148617396481d11df8547"}
Sep 09 20:53:39.201544 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:39.201487    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ltd4k" event={"ID":"c907cd72-cf97-4133-87f8-2e9bb7fb67a6","Type":"ContainerStarted","Data":"920fe3994d13984c77c941e69668eba383d7cfa21ca05585585b62090fc7e4c0"}
Sep 09 20:53:40.208062 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:40.208022    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ltd4k" event={"ID":"c907cd72-cf97-4133-87f8-2e9bb7fb67a6","Type":"ContainerStarted","Data":"9332075d91a618ab8b6312140e6d506c614abf4df3a926bda89f1a3b17a590e0"}
Sep 09 20:53:40.208488 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:40.208093    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ltd4k" event={"ID":"c907cd72-cf97-4133-87f8-2e9bb7fb67a6","Type":"ContainerStarted","Data":"7df28b08865576bee7b82ee4f93372d327b09f8c03fdca1465bae949858a5f8d"}
Sep 09 20:53:41.398390 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:41.398356    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:41.425889 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:41.425822    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ltd4k" podStartSLOduration=71.47842958 podStartE2EDuration="1m12.42579968s" totalImagesPullingTime="947.3701ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:40.238303375 +0000 UTC m=+71.908933074" watchObservedRunningTime="2026-09-09 20:53:41.42579968 +0000 UTC m=+73.096429381"
Sep 09 20:53:41.759125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:41.759029    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:41.759125 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:41.759111    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:41.764382 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:41.764357    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:42.221969 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.221929    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:53:42.341469 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.341436    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"]
Sep 09 20:53:42.343956 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.343932    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mrd8s"]
Sep 09 20:53:42.344089 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.344058    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.346276 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.346254    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.350021 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.349999    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 20:53:42.353349 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.353326    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 20:53:42.353758 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.353736    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vrjqf\""
Sep 09 20:53:42.353758 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.353736    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 20:53:42.354063 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.354047    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 20:53:42.354127 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.354095    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 09 20:53:42.354188 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.354129    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 20:53:42.355030 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.355009    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 09 20:53:42.355126 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.355027    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 09 20:53:42.355126 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.355060    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qq52t\""
Sep 09 20:53:42.355276 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.355261    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 20:53:42.373296 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.373267    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"]
Sep 09 20:53:42.441017 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.440985    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.441473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.441025    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.441473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.441053    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-node-exporter-textfile\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.441473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.441116    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-sys\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.441473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.441162    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-metrics-client-ca\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.441473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.441214    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc75s\" (UniqueName: \"kubernetes.io/projected/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-kube-api-access-xc75s\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.441473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.441260    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-node-exporter-accelerators-collector-config\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.441473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.441287    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-node-exporter-tls\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.441473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.441314    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-root\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.441473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.441335    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.441473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.441354    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-node-exporter-wtmp\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.441473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.441381    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.441473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.441412    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw94f\" (UniqueName: \"kubernetes.io/projected/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-kube-api-access-rw94f\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.441473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.441441    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.442048 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.441495    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.542777 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.542739    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-node-exporter-accelerators-collector-config\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.542777 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.542779    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-node-exporter-tls\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.543066 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.542798    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-root\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.543066 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.542818    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.543066 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.542835    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-node-exporter-wtmp\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.543066 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.542916    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-root\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.543066 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.542962    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.543066 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.543003    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-node-exporter-wtmp\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.543066 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.543012    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw94f\" (UniqueName: \"kubernetes.io/projected/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-kube-api-access-rw94f\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.543066 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.543046    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.543477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.543084    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.543477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.543130    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.543477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.543165    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.543477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.543187    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-node-exporter-textfile\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.543477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.543211    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-sys\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.543477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.543239    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-metrics-client-ca\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.543477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.543290    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc75s\" (UniqueName: \"kubernetes.io/projected/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-kube-api-access-xc75s\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.543477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.543334    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.543477 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.543461    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-sys\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.543910 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.543483    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-node-exporter-accelerators-collector-config\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.543910 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.543586    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.543910 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.543586    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-node-exporter-textfile\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.543910 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:42.543670    2585 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 09 20:53:42.543910 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:42.543729    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-kube-state-metrics-tls podName:af8e8d4d-b623-47d4-a5fb-9ca5fe30732c nodeName:}" failed. No retries permitted until 2026-09-09 20:53:43.043710441 +0000 UTC m=+74.714340134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-jkb4d" (UID: "af8e8d4d-b623-47d4-a5fb-9ca5fe30732c") : secret "kube-state-metrics-tls" not found
Sep 09 20:53:42.544132 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.543972    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-metrics-client-ca\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.544132 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.544119    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.545821 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.545794    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-node-exporter-tls\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.545979 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.545958    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.546118 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.546095    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.553743 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.553721    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc75s\" (UniqueName: \"kubernetes.io/projected/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-kube-api-access-xc75s\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:42.559462 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.559441    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw94f\" (UniqueName: \"kubernetes.io/projected/280a5f0f-cdd8-42d6-bf24-d1f6d9e73291-kube-api-access-rw94f\") pod \"node-exporter-mrd8s\" (UID: \"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291\") " pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:42.662660 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:42.662624    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mrd8s"
Sep 09 20:53:43.048005 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:43.047968    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:43.050563 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:43.050530    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af8e8d4d-b623-47d4-a5fb-9ca5fe30732c-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-jkb4d\" (UID: \"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:43.086296 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:53:43.086257    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod280a5f0f_cdd8_42d6_bf24_d1f6d9e73291.slice/crio-5b2872b36df20047fbb4db5fecb610a801e4031591094cd1d19a6b6194d8fa9f WatchSource:0}: Error finding container 5b2872b36df20047fbb4db5fecb610a801e4031591094cd1d19a6b6194d8fa9f: Status 404 returned error can't find the container with id 5b2872b36df20047fbb4db5fecb610a801e4031591094cd1d19a6b6194d8fa9f
Sep 09 20:53:43.222748 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:43.222705    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mrd8s" event={"ID":"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291","Type":"ContainerStarted","Data":"5b2872b36df20047fbb4db5fecb610a801e4031591094cd1d19a6b6194d8fa9f"}
Sep 09 20:53:43.224290 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:43.224258    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4ct29" event={"ID":"04fa9846-4123-477d-8c86-481d721ecc1e","Type":"ContainerStarted","Data":"fd39d917c3cb124f97b8872c1bf17a3a2892711ea379cd9a9a1c83c87de2af1f"}
Sep 09 20:53:43.224437 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:43.224366    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:53:43.225929 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:43.225904    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c28zc" event={"ID":"95b5dc36-8d5d-43d2-9444-5e870b1ca94b","Type":"ContainerStarted","Data":"5702355e99492df783897b5de8e1406c06d60eb95aad22e603c1581dd9786fdf"}
Sep 09 20:53:43.255985 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:43.255954    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"
Sep 09 20:53:43.264463 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:43.264417    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4ct29" podStartSLOduration=69.51135282 podStartE2EDuration="1m14.264401402s" totalImagesPullingTime="4.753048582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:43.263838558 +0000 UTC m=+74.934468259" watchObservedRunningTime="2026-09-09 20:53:43.264401402 +0000 UTC m=+74.935031102"
Sep 09 20:53:43.549024 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:43.548776    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-c28zc" podStartSLOduration=69.75149781 podStartE2EDuration="1m14.548718387s" totalImagesPullingTime="4.797220577s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:43.322005556 +0000 UTC m=+74.992635257" watchObservedRunningTime="2026-09-09 20:53:43.548718387 +0000 UTC m=+75.219348088"
Sep 09 20:53:43.549411 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:43.549020    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d"]
Sep 09 20:53:43.551227 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:53:43.551192    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf8e8d4d_b623_47d4_a5fb_9ca5fe30732c.slice/crio-fc747bc77353b0bf2c8a00c85cf12fe310fd6cc875196233df092faf4bab53e2 WatchSource:0}: Error finding container fc747bc77353b0bf2c8a00c85cf12fe310fd6cc875196233df092faf4bab53e2: Status 404 returned error can't find the container with id fc747bc77353b0bf2c8a00c85cf12fe310fd6cc875196233df092faf4bab53e2
Sep 09 20:53:44.231175 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:44.231091    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d" event={"ID":"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c","Type":"ContainerStarted","Data":"fc747bc77353b0bf2c8a00c85cf12fe310fd6cc875196233df092faf4bab53e2"}
Sep 09 20:53:44.232599 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:44.232575    2585 generic.go:356] "Generic (PLEG): container finished" podID="280a5f0f-cdd8-42d6-bf24-d1f6d9e73291" containerID="c96f16cf81d7448a5f5c59c6db0899088773a7ab25b8aa97166182a66608136a" exitCode=0
Sep 09 20:53:44.232708 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:44.232650    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mrd8s" event={"ID":"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291","Type":"ContainerDied","Data":"c96f16cf81d7448a5f5c59c6db0899088773a7ab25b8aa97166182a66608136a"}
Sep 09 20:53:45.238468 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:45.238370    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d" event={"ID":"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c","Type":"ContainerStarted","Data":"9d8e90e9f32654f011166668a399e1dd5ece43699cca0b120204771e5b23f45c"}
Sep 09 20:53:45.238468 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:45.238429    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d" event={"ID":"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c","Type":"ContainerStarted","Data":"ee8ad6c496b046beeef468bb682b5f2990c95815c355ef510a1b7b4921d87035"}
Sep 09 20:53:45.238468 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:45.238455    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d" event={"ID":"af8e8d4d-b623-47d4-a5fb-9ca5fe30732c","Type":"ContainerStarted","Data":"5b2b6ae7b71d6a6d5cc0f32b93c418199d28454acfa997a5909f60f31285975d"}
Sep 09 20:53:45.240579 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:45.240554    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mrd8s" event={"ID":"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291","Type":"ContainerStarted","Data":"8a96993c15fa3cdc9f524dc786bdc1d7687234d91daf329646de6aa701336914"}
Sep 09 20:53:45.240636 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:45.240605    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mrd8s" event={"ID":"280a5f0f-cdd8-42d6-bf24-d1f6d9e73291","Type":"ContainerStarted","Data":"e8754223fa1a2cc4c94c445a7018412cc51d787740a9c8d9227af0a48ddb60ff"}
Sep 09 20:53:45.272293 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:45.272235    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-jkb4d" podStartSLOduration=2.018039916 podStartE2EDuration="3.27221721s" totalImagesPullingTime="1.254177294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:45.27178917 +0000 UTC m=+76.942418889" watchObservedRunningTime="2026-09-09 20:53:45.27221721 +0000 UTC m=+76.942846910"
Sep 09 20:53:45.330681 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:45.330624    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mrd8s" podStartSLOduration=2.483402226 podStartE2EDuration="3.330610161s" totalImagesPullingTime="785.396113ms" totalInitContainerRuntime="61.811822ms" isStatefulPod=true podCreationTimestamp="2026-09-09 20:53:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:45.328784391 +0000 UTC m=+76.999414111" watchObservedRunningTime="2026-09-09 20:53:45.330610161 +0000 UTC m=+77.001239860"
Sep 09 20:53:46.770425 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:46.770387    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp"]
Sep 09 20:53:46.772402 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:46.772385    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp"
Sep 09 20:53:46.779491 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:46.779468    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-qkr8h\""
Sep 09 20:53:46.779979 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:46.779961    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 09 20:53:46.799430 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:46.799407    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp"]
Sep 09 20:53:46.882062 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:46.882021    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-288xv\" (UniqueName: \"kubernetes.io/projected/7c429400-a7e1-4059-afa6-4283be5c4966-kube-api-access-288xv\") pod \"monitoring-plugin-85bbbbbf4-nspsp\" (UID: \"7c429400-a7e1-4059-afa6-4283be5c4966\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp"
Sep 09 20:53:46.882226 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:46.882072    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c429400-a7e1-4059-afa6-4283be5c4966-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-nspsp\" (UID: \"7c429400-a7e1-4059-afa6-4283be5c4966\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp"
Sep 09 20:53:46.983229 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:46.983192    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-288xv\" (UniqueName: \"kubernetes.io/projected/7c429400-a7e1-4059-afa6-4283be5c4966-kube-api-access-288xv\") pod \"monitoring-plugin-85bbbbbf4-nspsp\" (UID: \"7c429400-a7e1-4059-afa6-4283be5c4966\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp"
Sep 09 20:53:46.983417 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:46.983243    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c429400-a7e1-4059-afa6-4283be5c4966-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-nspsp\" (UID: \"7c429400-a7e1-4059-afa6-4283be5c4966\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp"
Sep 09 20:53:46.983417 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:46.983347    2585 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 09 20:53:46.983417 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:46.983412    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c429400-a7e1-4059-afa6-4283be5c4966-monitoring-plugin-cert podName:7c429400-a7e1-4059-afa6-4283be5c4966 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:47.483396094 +0000 UTC m=+79.154025772 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/7c429400-a7e1-4059-afa6-4283be5c4966-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-nspsp" (UID: "7c429400-a7e1-4059-afa6-4283be5c4966") : secret "monitoring-plugin-cert" not found
Sep 09 20:53:47.021794 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:47.021720    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-288xv\" (UniqueName: \"kubernetes.io/projected/7c429400-a7e1-4059-afa6-4283be5c4966-kube-api-access-288xv\") pod \"monitoring-plugin-85bbbbbf4-nspsp\" (UID: \"7c429400-a7e1-4059-afa6-4283be5c4966\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp"
Sep 09 20:53:47.487693 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:47.487650    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c429400-a7e1-4059-afa6-4283be5c4966-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-nspsp\" (UID: \"7c429400-a7e1-4059-afa6-4283be5c4966\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp"
Sep 09 20:53:47.487907 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:47.487791    2585 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 09 20:53:47.487907 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:47.487846    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c429400-a7e1-4059-afa6-4283be5c4966-monitoring-plugin-cert podName:7c429400-a7e1-4059-afa6-4283be5c4966 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:48.487830203 +0000 UTC m=+80.158459881 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/7c429400-a7e1-4059-afa6-4283be5c4966-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-nspsp" (UID: "7c429400-a7e1-4059-afa6-4283be5c4966") : secret "monitoring-plugin-cert" not found
Sep 09 20:53:47.761407 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:47.761317    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bf8bb684-lx2vs"]
Sep 09 20:53:48.497163 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:48.497103    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c429400-a7e1-4059-afa6-4283be5c4966-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-nspsp\" (UID: \"7c429400-a7e1-4059-afa6-4283be5c4966\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp"
Sep 09 20:53:48.499641 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:48.499616    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c429400-a7e1-4059-afa6-4283be5c4966-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-nspsp\" (UID: \"7c429400-a7e1-4059-afa6-4283be5c4966\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp"
Sep 09 20:53:48.580543 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:48.580491    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp"
Sep 09 20:53:48.860430 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:48.860398    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp"]
Sep 09 20:53:48.864309 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:53:48.864271    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c429400_a7e1_4059_afa6_4283be5c4966.slice/crio-76ec6f4bb08116f8c37ba4aeaf0f1505028c392a523751a07b533e6dda86b515 WatchSource:0}: Error finding container 76ec6f4bb08116f8c37ba4aeaf0f1505028c392a523751a07b533e6dda86b515: Status 404 returned error can't find the container with id 76ec6f4bb08116f8c37ba4aeaf0f1505028c392a523751a07b533e6dda86b515
Sep 09 20:53:49.256912 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:49.256805    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp" event={"ID":"7c429400-a7e1-4059-afa6-4283be5c4966","Type":"ContainerStarted","Data":"76ec6f4bb08116f8c37ba4aeaf0f1505028c392a523751a07b533e6dda86b515"}
Sep 09 20:53:50.261565 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:50.261461    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp" event={"ID":"7c429400-a7e1-4059-afa6-4283be5c4966","Type":"ContainerStarted","Data":"a29220380d8e0090de9b3b194f61b6d1d47da6b40a1f55858dac499f8b5fa787"}
Sep 09 20:53:50.262027 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:50.261636    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp"
Sep 09 20:53:50.266281 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:50.266255    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp"
Sep 09 20:53:50.289838 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:50.289777    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-nspsp" podStartSLOduration=3.223332532 podStartE2EDuration="4.289761188s" totalImagesPullingTime="1.066428656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:50.288132877 +0000 UTC m=+81.958762576" watchObservedRunningTime="2026-09-09 20:53:50.289761188 +0000 UTC m=+81.960390888"
Sep 09 20:53:54.167760 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:54.167623    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d84dd5b4f-mjmz8"
Sep 09 20:53:56.411837 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.411788    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7d55d49d5-29jms" podUID="a6d02d8e-f8f3-4ade-9074-abd603c3e688" containerName="registry" containerID="cri-o://699cd3477c435ecadbbef94b4e5cdbd29a52f2be1886c5b99de8f62b960f4b72" gracePeriod=30
Sep 09 20:53:56.653721 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.653693    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:56.764812 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.764715    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-bound-sa-token\") pod \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") "
Sep 09 20:53:56.764812 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.764779    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6d02d8e-f8f3-4ade-9074-abd603c3e688-trusted-ca\") pod \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") "
Sep 09 20:53:56.764812 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.764814    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6d02d8e-f8f3-4ade-9074-abd603c3e688-installation-pull-secrets\") pod \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") "
Sep 09 20:53:56.765089 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.764847    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mw2g\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-kube-api-access-7mw2g\") pod \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") "
Sep 09 20:53:56.765089 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.764870    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls\") pod \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") "
Sep 09 20:53:56.765089 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.764943    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-certificates\") pod \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") "
Sep 09 20:53:56.765089 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.764970    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6d02d8e-f8f3-4ade-9074-abd603c3e688-ca-trust-extracted\") pod \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") "
Sep 09 20:53:56.765089 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.765008    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6d02d8e-f8f3-4ade-9074-abd603c3e688-image-registry-private-configuration\") pod \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\" (UID: \"a6d02d8e-f8f3-4ade-9074-abd603c3e688\") "
Sep 09 20:53:56.765529 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.765467    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a6d02d8e-f8f3-4ade-9074-abd603c3e688" (UID: "a6d02d8e-f8f3-4ade-9074-abd603c3e688"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:53:56.765644 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.765580    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6d02d8e-f8f3-4ade-9074-abd603c3e688-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a6d02d8e-f8f3-4ade-9074-abd603c3e688" (UID: "a6d02d8e-f8f3-4ade-9074-abd603c3e688"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:53:56.767650 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.767604    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a6d02d8e-f8f3-4ade-9074-abd603c3e688" (UID: "a6d02d8e-f8f3-4ade-9074-abd603c3e688"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:53:56.767764 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.767674    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a6d02d8e-f8f3-4ade-9074-abd603c3e688" (UID: "a6d02d8e-f8f3-4ade-9074-abd603c3e688"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:53:56.767764 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.767756    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-kube-api-access-7mw2g" (OuterVolumeSpecName: "kube-api-access-7mw2g") pod "a6d02d8e-f8f3-4ade-9074-abd603c3e688" (UID: "a6d02d8e-f8f3-4ade-9074-abd603c3e688"). InnerVolumeSpecName "kube-api-access-7mw2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:53:56.767852 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.767773    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6d02d8e-f8f3-4ade-9074-abd603c3e688-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a6d02d8e-f8f3-4ade-9074-abd603c3e688" (UID: "a6d02d8e-f8f3-4ade-9074-abd603c3e688"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:53:56.767852 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.767839    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6d02d8e-f8f3-4ade-9074-abd603c3e688-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a6d02d8e-f8f3-4ade-9074-abd603c3e688" (UID: "a6d02d8e-f8f3-4ade-9074-abd603c3e688"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:53:56.774380 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.774349    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6d02d8e-f8f3-4ade-9074-abd603c3e688-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a6d02d8e-f8f3-4ade-9074-abd603c3e688" (UID: "a6d02d8e-f8f3-4ade-9074-abd603c3e688"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:53:56.866131 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.866097    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mw2g\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-kube-api-access-7mw2g\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:53:56.866131 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.866129    2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-tls\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:53:56.866131 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.866139    2585 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6d02d8e-f8f3-4ade-9074-abd603c3e688-registry-certificates\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:53:56.866356 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.866148    2585 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6d02d8e-f8f3-4ade-9074-abd603c3e688-ca-trust-extracted\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:53:56.866356 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.866157    2585 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6d02d8e-f8f3-4ade-9074-abd603c3e688-image-registry-private-configuration\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:53:56.866356 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.866167    2585 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6d02d8e-f8f3-4ade-9074-abd603c3e688-bound-sa-token\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:53:56.866356 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.866175    2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6d02d8e-f8f3-4ade-9074-abd603c3e688-trusted-ca\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:53:56.866356 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:56.866183    2585 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6d02d8e-f8f3-4ade-9074-abd603c3e688-installation-pull-secrets\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:53:57.288132 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:57.288093    2585 generic.go:356] "Generic (PLEG): container finished" podID="a6d02d8e-f8f3-4ade-9074-abd603c3e688" containerID="699cd3477c435ecadbbef94b4e5cdbd29a52f2be1886c5b99de8f62b960f4b72" exitCode=0
Sep 09 20:53:57.288287 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:57.288143    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d55d49d5-29jms"
Sep 09 20:53:57.288287 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:57.288179    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d55d49d5-29jms" event={"ID":"a6d02d8e-f8f3-4ade-9074-abd603c3e688","Type":"ContainerDied","Data":"699cd3477c435ecadbbef94b4e5cdbd29a52f2be1886c5b99de8f62b960f4b72"}
Sep 09 20:53:57.288287 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:57.288237    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d55d49d5-29jms" event={"ID":"a6d02d8e-f8f3-4ade-9074-abd603c3e688","Type":"ContainerDied","Data":"1cf4dca04a1e3d0ac220666aea22b50da79fca33699434212a0cd8dafb9db132"}
Sep 09 20:53:57.288287 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:57.288276    2585 scope.go:117] "RemoveContainer" containerID="699cd3477c435ecadbbef94b4e5cdbd29a52f2be1886c5b99de8f62b960f4b72"
Sep 09 20:53:57.296995 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:57.296972    2585 scope.go:117] "RemoveContainer" containerID="699cd3477c435ecadbbef94b4e5cdbd29a52f2be1886c5b99de8f62b960f4b72"
Sep 09 20:53:57.297276 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:53:57.297255    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"699cd3477c435ecadbbef94b4e5cdbd29a52f2be1886c5b99de8f62b960f4b72\": container with ID starting with 699cd3477c435ecadbbef94b4e5cdbd29a52f2be1886c5b99de8f62b960f4b72 not found: ID does not exist" containerID="699cd3477c435ecadbbef94b4e5cdbd29a52f2be1886c5b99de8f62b960f4b72"
Sep 09 20:53:57.297331 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:57.297284    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"699cd3477c435ecadbbef94b4e5cdbd29a52f2be1886c5b99de8f62b960f4b72"} err="failed to get container status \"699cd3477c435ecadbbef94b4e5cdbd29a52f2be1886c5b99de8f62b960f4b72\": rpc error: code = NotFound desc = could not find container \"699cd3477c435ecadbbef94b4e5cdbd29a52f2be1886c5b99de8f62b960f4b72\": container with ID starting with 699cd3477c435ecadbbef94b4e5cdbd29a52f2be1886c5b99de8f62b960f4b72 not found: ID does not exist"
Sep 09 20:53:57.318846 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:57.318811    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d55d49d5-29jms"]
Sep 09 20:53:57.326635 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:57.326607    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7d55d49d5-29jms"]
Sep 09 20:53:58.798423 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:53:58.798378    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6d02d8e-f8f3-4ade-9074-abd603c3e688" path="/var/lib/kubelet/pods/a6d02d8e-f8f3-4ade-9074-abd603c3e688/volumes"
Sep 09 20:54:12.781107 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:12.781061    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6bf8bb684-lx2vs" podUID="2724eaa6-cd7a-4ede-9d66-49417b3fb015" containerName="console" containerID="cri-o://194f3cf9aebedb6893ba5d6b782f62514f9e0574b814bed34610c824be975375" gracePeriod=15
Sep 09 20:54:13.348401 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.348369    2585 generic.go:356] "Generic (PLEG): container finished" podID="2724eaa6-cd7a-4ede-9d66-49417b3fb015" containerID="194f3cf9aebedb6893ba5d6b782f62514f9e0574b814bed34610c824be975375" exitCode=0
Sep 09 20:54:13.348579 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.348424    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf8bb684-lx2vs" event={"ID":"2724eaa6-cd7a-4ede-9d66-49417b3fb015","Type":"ContainerDied","Data":"194f3cf9aebedb6893ba5d6b782f62514f9e0574b814bed34610c824be975375"}
Sep 09 20:54:13.562887 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.562837    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:54:13.700091 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.699993    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-config\") pod \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") "
Sep 09 20:54:13.700091 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.700061    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-oauth-serving-cert\") pod \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") "
Sep 09 20:54:13.700091 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.700093    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-oauth-config\") pod \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") "
Sep 09 20:54:13.700320 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.700135    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-service-ca\") pod \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") "
Sep 09 20:54:13.700320 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.700166    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-serving-cert\") pod \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") "
Sep 09 20:54:13.700320 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.700202    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4zfh\" (UniqueName: \"kubernetes.io/projected/2724eaa6-cd7a-4ede-9d66-49417b3fb015-kube-api-access-d4zfh\") pod \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\" (UID: \"2724eaa6-cd7a-4ede-9d66-49417b3fb015\") "
Sep 09 20:54:13.700510 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.700485    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-config" (OuterVolumeSpecName: "console-config") pod "2724eaa6-cd7a-4ede-9d66-49417b3fb015" (UID: "2724eaa6-cd7a-4ede-9d66-49417b3fb015"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:13.700670 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.700643    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2724eaa6-cd7a-4ede-9d66-49417b3fb015" (UID: "2724eaa6-cd7a-4ede-9d66-49417b3fb015"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:13.700733 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.700645    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-service-ca" (OuterVolumeSpecName: "service-ca") pod "2724eaa6-cd7a-4ede-9d66-49417b3fb015" (UID: "2724eaa6-cd7a-4ede-9d66-49417b3fb015"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:13.702570 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.702540    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2724eaa6-cd7a-4ede-9d66-49417b3fb015" (UID: "2724eaa6-cd7a-4ede-9d66-49417b3fb015"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:54:13.702570 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.702543    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2724eaa6-cd7a-4ede-9d66-49417b3fb015-kube-api-access-d4zfh" (OuterVolumeSpecName: "kube-api-access-d4zfh") pod "2724eaa6-cd7a-4ede-9d66-49417b3fb015" (UID: "2724eaa6-cd7a-4ede-9d66-49417b3fb015"). InnerVolumeSpecName "kube-api-access-d4zfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:54:13.702704 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.702596    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2724eaa6-cd7a-4ede-9d66-49417b3fb015" (UID: "2724eaa6-cd7a-4ede-9d66-49417b3fb015"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:54:13.801736 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.801693    2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-config\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:54:13.801736 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.801727    2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-oauth-serving-cert\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:54:13.801736 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.801737    2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-oauth-config\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:54:13.801736 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.801746    2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2724eaa6-cd7a-4ede-9d66-49417b3fb015-service-ca\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:54:13.802253 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.801754    2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2724eaa6-cd7a-4ede-9d66-49417b3fb015-console-serving-cert\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:54:13.802253 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:13.801762    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4zfh\" (UniqueName: \"kubernetes.io/projected/2724eaa6-cd7a-4ede-9d66-49417b3fb015-kube-api-access-d4zfh\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:54:14.235732 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:14.235701    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4ct29"
Sep 09 20:54:14.353431 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:14.353398    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf8bb684-lx2vs"
Sep 09 20:54:14.353603 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:14.353401    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf8bb684-lx2vs" event={"ID":"2724eaa6-cd7a-4ede-9d66-49417b3fb015","Type":"ContainerDied","Data":"22404cf3a11c2bcffb3cb87f53de39110df12326646b7972bd8fcc6831f9bd50"}
Sep 09 20:54:14.353603 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:14.353544    2585 scope.go:117] "RemoveContainer" containerID="194f3cf9aebedb6893ba5d6b782f62514f9e0574b814bed34610c824be975375"
Sep 09 20:54:14.375637 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:14.375605    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bf8bb684-lx2vs"]
Sep 09 20:54:14.383003 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:14.382976    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bf8bb684-lx2vs"]
Sep 09 20:54:14.798312 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:54:14.798279    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2724eaa6-cd7a-4ede-9d66-49417b3fb015" path="/var/lib/kubelet/pods/2724eaa6-cd7a-4ede-9d66-49417b3fb015/volumes"
Sep 09 20:55:09.858626 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.858588    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7777f547cb-2s94z"]
Sep 09 20:55:09.859092 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.858829    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2724eaa6-cd7a-4ede-9d66-49417b3fb015" containerName="console"
Sep 09 20:55:09.859092 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.858839    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2724eaa6-cd7a-4ede-9d66-49417b3fb015" containerName="console"
Sep 09 20:55:09.859092 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.858849    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6d02d8e-f8f3-4ade-9074-abd603c3e688" containerName="registry"
Sep 09 20:55:09.859092 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.858854    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6d02d8e-f8f3-4ade-9074-abd603c3e688" containerName="registry"
Sep 09 20:55:09.859092 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.858919    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2724eaa6-cd7a-4ede-9d66-49417b3fb015" containerName="console"
Sep 09 20:55:09.859092 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.858927    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6d02d8e-f8f3-4ade-9074-abd603c3e688" containerName="registry"
Sep 09 20:55:09.861653 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.861636    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.864740 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.864714    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 09 20:55:09.864869 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.864787    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 09 20:55:09.864971 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.864865    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 09 20:55:09.866127 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.866086    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 09 20:55:09.866127 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.866103    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 09 20:55:09.866127 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.866086    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 09 20:55:09.866345 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.866167    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kgh55\""
Sep 09 20:55:09.866345 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.866191    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 09 20:55:09.871239 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.871215    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 09 20:55:09.872712 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.872690    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7777f547cb-2s94z"]
Sep 09 20:55:09.892933 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.892895    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-trusted-ca-bundle\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.893078 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.892967    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-console-config\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.893078 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.892990    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-service-ca\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.893078 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.893010    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-oauth-serving-cert\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.893078 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.893031    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df87356d-eae8-4194-90eb-630f6280346a-console-oauth-config\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.893206 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.893107    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-246cx\" (UniqueName: \"kubernetes.io/projected/df87356d-eae8-4194-90eb-630f6280346a-kube-api-access-246cx\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.893206 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.893133    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df87356d-eae8-4194-90eb-630f6280346a-console-serving-cert\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.993907 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.993838    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-trusted-ca-bundle\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.994121 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.993937    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-console-config\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.994121 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.993963    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-service-ca\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.994121 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.993984    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-oauth-serving-cert\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.994121 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.994005    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df87356d-eae8-4194-90eb-630f6280346a-console-oauth-config\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.994121 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.994023    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-246cx\" (UniqueName: \"kubernetes.io/projected/df87356d-eae8-4194-90eb-630f6280346a-kube-api-access-246cx\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.994121 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.994047    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df87356d-eae8-4194-90eb-630f6280346a-console-serving-cert\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.994826 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.994799    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-oauth-serving-cert\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.994974 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.994821    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-console-config\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.994974 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.994828    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-service-ca\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.994974 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.994921    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-trusted-ca-bundle\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.996501 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.996474    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df87356d-eae8-4194-90eb-630f6280346a-console-serving-cert\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:09.996652 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:09.996632    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df87356d-eae8-4194-90eb-630f6280346a-console-oauth-config\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:10.003748 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:10.003721    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-246cx\" (UniqueName: \"kubernetes.io/projected/df87356d-eae8-4194-90eb-630f6280346a-kube-api-access-246cx\") pod \"console-7777f547cb-2s94z\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") " pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:10.171343 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:10.171250    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:10.446317 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:10.446243    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7777f547cb-2s94z"]
Sep 09 20:55:10.449395 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:55:10.449369    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf87356d_eae8_4194_90eb_630f6280346a.slice/crio-02a3e9141f1369f733afd7667418a3b6cee15b11ff5e202d142cf6aed0e3636c WatchSource:0}: Error finding container 02a3e9141f1369f733afd7667418a3b6cee15b11ff5e202d142cf6aed0e3636c: Status 404 returned error can't find the container with id 02a3e9141f1369f733afd7667418a3b6cee15b11ff5e202d142cf6aed0e3636c
Sep 09 20:55:10.546807 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:10.546765    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7777f547cb-2s94z" event={"ID":"df87356d-eae8-4194-90eb-630f6280346a","Type":"ContainerStarted","Data":"c7c86e6506d57cef50f824445e77f9d7ef2fb7d207ac7ae89dc0d5e70a5fdefe"}
Sep 09 20:55:10.546974 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:10.546819    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7777f547cb-2s94z" event={"ID":"df87356d-eae8-4194-90eb-630f6280346a","Type":"ContainerStarted","Data":"02a3e9141f1369f733afd7667418a3b6cee15b11ff5e202d142cf6aed0e3636c"}
Sep 09 20:55:10.576750 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:10.576690    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7777f547cb-2s94z" podStartSLOduration=1.576672193 podStartE2EDuration="1.576672193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:55:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:10.573871157 +0000 UTC m=+162.244500912" watchObservedRunningTime="2026-09-09 20:55:10.576672193 +0000 UTC m=+162.247301892"
Sep 09 20:55:20.171706 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.171661    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:20.172265 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.171742    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:20.176319 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.176295    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:20.586001 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.585969    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:20.689863 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.689828    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-554c4ccdcf-2sgl8"]
Sep 09 20:55:20.691952 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.691932    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.703278 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.703246    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-554c4ccdcf-2sgl8"]
Sep 09 20:55:20.780820 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.780779    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c48d054-b99d-4d09-b731-07260dea1100-console-serving-cert\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.784926 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.781234    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c48d054-b99d-4d09-b731-07260dea1100-console-oauth-config\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.784926 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.781375    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-console-config\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.784926 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.781424    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-service-ca\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.784926 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.781474    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-trusted-ca-bundle\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.784926 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.781506    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktgp4\" (UniqueName: \"kubernetes.io/projected/4c48d054-b99d-4d09-b731-07260dea1100-kube-api-access-ktgp4\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.784926 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.781535    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-oauth-serving-cert\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.882941 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.882820    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c48d054-b99d-4d09-b731-07260dea1100-console-serving-cert\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.882941 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.882872    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c48d054-b99d-4d09-b731-07260dea1100-console-oauth-config\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.883150 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.883029    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-console-config\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.883150 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.883077    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-service-ca\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.883150 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.883122    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-trusted-ca-bundle\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.883310 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.883148    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktgp4\" (UniqueName: \"kubernetes.io/projected/4c48d054-b99d-4d09-b731-07260dea1100-kube-api-access-ktgp4\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.883310 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.883181    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-oauth-serving-cert\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.883773 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.883748    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-console-config\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.883946 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.883774    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-service-ca\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.884012 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.883991    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-oauth-serving-cert\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.884130 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.884106    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-trusted-ca-bundle\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.885534 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.885508    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c48d054-b99d-4d09-b731-07260dea1100-console-oauth-config\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.885610 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.885567    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c48d054-b99d-4d09-b731-07260dea1100-console-serving-cert\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:20.892406 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:20.892366    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktgp4\" (UniqueName: \"kubernetes.io/projected/4c48d054-b99d-4d09-b731-07260dea1100-kube-api-access-ktgp4\") pod \"console-554c4ccdcf-2sgl8\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") " pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:21.000737 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:21.000691    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:21.278631 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:21.278604    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-554c4ccdcf-2sgl8"]
Sep 09 20:55:21.278956 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:55:21.278725    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c48d054_b99d_4d09_b731_07260dea1100.slice/crio-4d77facc7d2a17ac54480bfe503fd89c0750a5385db951215c18d97ca087eef2 WatchSource:0}: Error finding container 4d77facc7d2a17ac54480bfe503fd89c0750a5385db951215c18d97ca087eef2: Status 404 returned error can't find the container with id 4d77facc7d2a17ac54480bfe503fd89c0750a5385db951215c18d97ca087eef2
Sep 09 20:55:21.586856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:21.586820    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554c4ccdcf-2sgl8" event={"ID":"4c48d054-b99d-4d09-b731-07260dea1100","Type":"ContainerStarted","Data":"73bb0cc9d44b4f1ed60b2c243aa111159f04ddbca174dd6337835bab6f06a903"}
Sep 09 20:55:21.587044 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:21.586873    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554c4ccdcf-2sgl8" event={"ID":"4c48d054-b99d-4d09-b731-07260dea1100","Type":"ContainerStarted","Data":"4d77facc7d2a17ac54480bfe503fd89c0750a5385db951215c18d97ca087eef2"}
Sep 09 20:55:21.604843 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:21.604789    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-554c4ccdcf-2sgl8" podStartSLOduration=1.6047709289999998 podStartE2EDuration="1.604770929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:21.60431348 +0000 UTC m=+173.274943180" watchObservedRunningTime="2026-09-09 20:55:21.604770929 +0000 UTC m=+173.275400630"
Sep 09 20:55:31.001827 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:31.001782    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:31.002244 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:31.001849    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:31.006359 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:31.006336    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:31.625286 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:31.625258    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:55:31.679633 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:31.679600    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7777f547cb-2s94z"]
Sep 09 20:55:56.700118 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:56.700056    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7777f547cb-2s94z" podUID="df87356d-eae8-4194-90eb-630f6280346a" containerName="console" containerID="cri-o://c7c86e6506d57cef50f824445e77f9d7ef2fb7d207ac7ae89dc0d5e70a5fdefe" gracePeriod=15
Sep 09 20:55:56.946672 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:56.946645    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:56.958127 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:56.958059    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df87356d-eae8-4194-90eb-630f6280346a-console-oauth-config\") pod \"df87356d-eae8-4194-90eb-630f6280346a\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") "
Sep 09 20:55:56.958127 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:56.958102    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-service-ca\") pod \"df87356d-eae8-4194-90eb-630f6280346a\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") "
Sep 09 20:55:56.958311 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:56.958139    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df87356d-eae8-4194-90eb-630f6280346a-console-serving-cert\") pod \"df87356d-eae8-4194-90eb-630f6280346a\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") "
Sep 09 20:55:56.958311 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:56.958170    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-trusted-ca-bundle\") pod \"df87356d-eae8-4194-90eb-630f6280346a\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") "
Sep 09 20:55:56.958311 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:56.958209    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-246cx\" (UniqueName: \"kubernetes.io/projected/df87356d-eae8-4194-90eb-630f6280346a-kube-api-access-246cx\") pod \"df87356d-eae8-4194-90eb-630f6280346a\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") "
Sep 09 20:55:56.958311 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:56.958244    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-console-config\") pod \"df87356d-eae8-4194-90eb-630f6280346a\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") "
Sep 09 20:55:56.958589 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:56.958555    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-service-ca" (OuterVolumeSpecName: "service-ca") pod "df87356d-eae8-4194-90eb-630f6280346a" (UID: "df87356d-eae8-4194-90eb-630f6280346a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:55:56.958713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:56.958620    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "df87356d-eae8-4194-90eb-630f6280346a" (UID: "df87356d-eae8-4194-90eb-630f6280346a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:55:56.958713 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:56.958670    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-console-config" (OuterVolumeSpecName: "console-config") pod "df87356d-eae8-4194-90eb-630f6280346a" (UID: "df87356d-eae8-4194-90eb-630f6280346a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:55:56.960237 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:56.960206    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df87356d-eae8-4194-90eb-630f6280346a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "df87356d-eae8-4194-90eb-630f6280346a" (UID: "df87356d-eae8-4194-90eb-630f6280346a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:55:56.960324 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:56.960241    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df87356d-eae8-4194-90eb-630f6280346a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "df87356d-eae8-4194-90eb-630f6280346a" (UID: "df87356d-eae8-4194-90eb-630f6280346a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:55:56.960324 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:56.960307    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df87356d-eae8-4194-90eb-630f6280346a-kube-api-access-246cx" (OuterVolumeSpecName: "kube-api-access-246cx") pod "df87356d-eae8-4194-90eb-630f6280346a" (UID: "df87356d-eae8-4194-90eb-630f6280346a"). InnerVolumeSpecName "kube-api-access-246cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:55:57.059497 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.059463    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-oauth-serving-cert\") pod \"df87356d-eae8-4194-90eb-630f6280346a\" (UID: \"df87356d-eae8-4194-90eb-630f6280346a\") "
Sep 09 20:55:57.059677 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.059622    2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df87356d-eae8-4194-90eb-630f6280346a-console-oauth-config\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:55:57.059677 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.059633    2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-service-ca\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:55:57.059677 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.059642    2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df87356d-eae8-4194-90eb-630f6280346a-console-serving-cert\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:55:57.059677 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.059651    2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-trusted-ca-bundle\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:55:57.059677 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.059660    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-246cx\" (UniqueName: \"kubernetes.io/projected/df87356d-eae8-4194-90eb-630f6280346a-kube-api-access-246cx\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:55:57.059677 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.059669    2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-console-config\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:55:57.059857 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.059787    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "df87356d-eae8-4194-90eb-630f6280346a" (UID: "df87356d-eae8-4194-90eb-630f6280346a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:55:57.161030 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.160988    2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df87356d-eae8-4194-90eb-630f6280346a-oauth-serving-cert\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:55:57.711655 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.711621    2585 generic.go:356] "Generic (PLEG): container finished" podID="df87356d-eae8-4194-90eb-630f6280346a" containerID="c7c86e6506d57cef50f824445e77f9d7ef2fb7d207ac7ae89dc0d5e70a5fdefe" exitCode=0
Sep 09 20:55:57.712103 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.711679    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7777f547cb-2s94z"
Sep 09 20:55:57.712103 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.711680    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7777f547cb-2s94z" event={"ID":"df87356d-eae8-4194-90eb-630f6280346a","Type":"ContainerDied","Data":"c7c86e6506d57cef50f824445e77f9d7ef2fb7d207ac7ae89dc0d5e70a5fdefe"}
Sep 09 20:55:57.712103 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.711829    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7777f547cb-2s94z" event={"ID":"df87356d-eae8-4194-90eb-630f6280346a","Type":"ContainerDied","Data":"02a3e9141f1369f733afd7667418a3b6cee15b11ff5e202d142cf6aed0e3636c"}
Sep 09 20:55:57.712103 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.711859    2585 scope.go:117] "RemoveContainer" containerID="c7c86e6506d57cef50f824445e77f9d7ef2fb7d207ac7ae89dc0d5e70a5fdefe"
Sep 09 20:55:57.720780 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.720764    2585 scope.go:117] "RemoveContainer" containerID="c7c86e6506d57cef50f824445e77f9d7ef2fb7d207ac7ae89dc0d5e70a5fdefe"
Sep 09 20:55:57.721077 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:55:57.721056    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7c86e6506d57cef50f824445e77f9d7ef2fb7d207ac7ae89dc0d5e70a5fdefe\": container with ID starting with c7c86e6506d57cef50f824445e77f9d7ef2fb7d207ac7ae89dc0d5e70a5fdefe not found: ID does not exist" containerID="c7c86e6506d57cef50f824445e77f9d7ef2fb7d207ac7ae89dc0d5e70a5fdefe"
Sep 09 20:55:57.721133 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.721086    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7c86e6506d57cef50f824445e77f9d7ef2fb7d207ac7ae89dc0d5e70a5fdefe"} err="failed to get container status \"c7c86e6506d57cef50f824445e77f9d7ef2fb7d207ac7ae89dc0d5e70a5fdefe\": rpc error: code = NotFound desc = could not find container \"c7c86e6506d57cef50f824445e77f9d7ef2fb7d207ac7ae89dc0d5e70a5fdefe\": container with ID starting with c7c86e6506d57cef50f824445e77f9d7ef2fb7d207ac7ae89dc0d5e70a5fdefe not found: ID does not exist"
Sep 09 20:55:57.734670 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.734641    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7777f547cb-2s94z"]
Sep 09 20:55:57.741793 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:57.741761    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7777f547cb-2s94z"]
Sep 09 20:55:58.798256 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:55:58.798224    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df87356d-eae8-4194-90eb-630f6280346a" path="/var/lib/kubelet/pods/df87356d-eae8-4194-90eb-630f6280346a/volumes"
Sep 09 20:56:56.792793 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:56.792702    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-579c8958f7-99rvq"]
Sep 09 20:56:56.793281 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:56.793110    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df87356d-eae8-4194-90eb-630f6280346a" containerName="console"
Sep 09 20:56:56.793281 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:56.793128    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="df87356d-eae8-4194-90eb-630f6280346a" containerName="console"
Sep 09 20:56:56.793281 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:56.793195    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="df87356d-eae8-4194-90eb-630f6280346a" containerName="console"
Sep 09 20:56:56.796095 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:56.796070    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-99rvq"
Sep 09 20:56:56.800194 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:56.800169    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Sep 09 20:56:56.800620 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:56.800599    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 09 20:56:56.801544 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:56.801523    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-v6g59\""
Sep 09 20:56:56.801544 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:56.801544    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 09 20:56:56.801698 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:56.801600    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 09 20:56:56.802275 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:56.802259    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Sep 09 20:56:56.809294 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:56.809274    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-99rvq"]
Sep 09 20:56:56.902077 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:56.902049    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/05fffb82-92b8-4d04-a29c-fead867763fa-certificates\") pod \"keda-operator-579c8958f7-99rvq\" (UID: \"05fffb82-92b8-4d04-a29c-fead867763fa\") " pod="openshift-keda/keda-operator-579c8958f7-99rvq"
Sep 09 20:56:56.902254 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:56.902092    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/05fffb82-92b8-4d04-a29c-fead867763fa-cabundle0\") pod \"keda-operator-579c8958f7-99rvq\" (UID: \"05fffb82-92b8-4d04-a29c-fead867763fa\") " pod="openshift-keda/keda-operator-579c8958f7-99rvq"
Sep 09 20:56:56.902254 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:56.902139    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rcst\" (UniqueName: \"kubernetes.io/projected/05fffb82-92b8-4d04-a29c-fead867763fa-kube-api-access-9rcst\") pod \"keda-operator-579c8958f7-99rvq\" (UID: \"05fffb82-92b8-4d04-a29c-fead867763fa\") " pod="openshift-keda/keda-operator-579c8958f7-99rvq"
Sep 09 20:56:57.002742 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.002700    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/05fffb82-92b8-4d04-a29c-fead867763fa-cabundle0\") pod \"keda-operator-579c8958f7-99rvq\" (UID: \"05fffb82-92b8-4d04-a29c-fead867763fa\") " pod="openshift-keda/keda-operator-579c8958f7-99rvq"
Sep 09 20:56:57.002961 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.002758    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rcst\" (UniqueName: \"kubernetes.io/projected/05fffb82-92b8-4d04-a29c-fead867763fa-kube-api-access-9rcst\") pod \"keda-operator-579c8958f7-99rvq\" (UID: \"05fffb82-92b8-4d04-a29c-fead867763fa\") " pod="openshift-keda/keda-operator-579c8958f7-99rvq"
Sep 09 20:56:57.002961 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.002795    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/05fffb82-92b8-4d04-a29c-fead867763fa-certificates\") pod \"keda-operator-579c8958f7-99rvq\" (UID: \"05fffb82-92b8-4d04-a29c-fead867763fa\") " pod="openshift-keda/keda-operator-579c8958f7-99rvq"
Sep 09 20:56:57.003089 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:56:57.002976    2585 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Sep 09 20:56:57.003089 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:56:57.002998    2585 secret.go:281] references non-existent secret key: ca.crt
Sep 09 20:56:57.003089 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:56:57.003006    2585 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 09 20:56:57.003089 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:56:57.003018    2585 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-99rvq: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Sep 09 20:56:57.003089 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:56:57.003066    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05fffb82-92b8-4d04-a29c-fead867763fa-certificates podName:05fffb82-92b8-4d04-a29c-fead867763fa nodeName:}" failed. No retries permitted until 2026-09-09 20:56:57.503049426 +0000 UTC m=+269.173679108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/05fffb82-92b8-4d04-a29c-fead867763fa-certificates") pod "keda-operator-579c8958f7-99rvq" (UID: "05fffb82-92b8-4d04-a29c-fead867763fa") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Sep 09 20:56:57.003356 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.003337    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/05fffb82-92b8-4d04-a29c-fead867763fa-cabundle0\") pod \"keda-operator-579c8958f7-99rvq\" (UID: \"05fffb82-92b8-4d04-a29c-fead867763fa\") " pod="openshift-keda/keda-operator-579c8958f7-99rvq"
Sep 09 20:56:57.036483 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.036457    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rcst\" (UniqueName: \"kubernetes.io/projected/05fffb82-92b8-4d04-a29c-fead867763fa-kube-api-access-9rcst\") pod \"keda-operator-579c8958f7-99rvq\" (UID: \"05fffb82-92b8-4d04-a29c-fead867763fa\") " pod="openshift-keda/keda-operator-579c8958f7-99rvq"
Sep 09 20:56:57.507609 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.507573    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/05fffb82-92b8-4d04-a29c-fead867763fa-certificates\") pod \"keda-operator-579c8958f7-99rvq\" (UID: \"05fffb82-92b8-4d04-a29c-fead867763fa\") " pod="openshift-keda/keda-operator-579c8958f7-99rvq"
Sep 09 20:56:57.510137 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.510110    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/05fffb82-92b8-4d04-a29c-fead867763fa-certificates\") pod \"keda-operator-579c8958f7-99rvq\" (UID: \"05fffb82-92b8-4d04-a29c-fead867763fa\") " pod="openshift-keda/keda-operator-579c8958f7-99rvq"
Sep 09 20:56:57.564917 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.564867    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-d4w2c"]
Sep 09 20:56:57.568138 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.568119    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-d4w2c"
Sep 09 20:56:57.570984 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.570960    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Sep 09 20:56:57.582036 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.582009    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-d4w2c"]
Sep 09 20:56:57.706625 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.706589    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-99rvq"
Sep 09 20:56:57.708903 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.708864    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rczhd\" (UniqueName: \"kubernetes.io/projected/b8adf056-1c9d-4dd0-9428-4771e32c43ee-kube-api-access-rczhd\") pod \"keda-admission-6c4d864b86-d4w2c\" (UID: \"b8adf056-1c9d-4dd0-9428-4771e32c43ee\") " pod="openshift-keda/keda-admission-6c4d864b86-d4w2c"
Sep 09 20:56:57.708979 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.708921    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/b8adf056-1c9d-4dd0-9428-4771e32c43ee-certificates\") pod \"keda-admission-6c4d864b86-d4w2c\" (UID: \"b8adf056-1c9d-4dd0-9428-4771e32c43ee\") " pod="openshift-keda/keda-admission-6c4d864b86-d4w2c"
Sep 09 20:56:57.809556 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.809522    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rczhd\" (UniqueName: \"kubernetes.io/projected/b8adf056-1c9d-4dd0-9428-4771e32c43ee-kube-api-access-rczhd\") pod \"keda-admission-6c4d864b86-d4w2c\" (UID: \"b8adf056-1c9d-4dd0-9428-4771e32c43ee\") " pod="openshift-keda/keda-admission-6c4d864b86-d4w2c"
Sep 09 20:56:57.809933 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.809565    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/b8adf056-1c9d-4dd0-9428-4771e32c43ee-certificates\") pod \"keda-admission-6c4d864b86-d4w2c\" (UID: \"b8adf056-1c9d-4dd0-9428-4771e32c43ee\") " pod="openshift-keda/keda-admission-6c4d864b86-d4w2c"
Sep 09 20:56:57.812167 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.812136    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/b8adf056-1c9d-4dd0-9428-4771e32c43ee-certificates\") pod \"keda-admission-6c4d864b86-d4w2c\" (UID: \"b8adf056-1c9d-4dd0-9428-4771e32c43ee\") " pod="openshift-keda/keda-admission-6c4d864b86-d4w2c"
Sep 09 20:56:57.818611 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.818588    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rczhd\" (UniqueName: \"kubernetes.io/projected/b8adf056-1c9d-4dd0-9428-4771e32c43ee-kube-api-access-rczhd\") pod \"keda-admission-6c4d864b86-d4w2c\" (UID: \"b8adf056-1c9d-4dd0-9428-4771e32c43ee\") " pod="openshift-keda/keda-admission-6c4d864b86-d4w2c"
Sep 09 20:56:57.878537 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.878501    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-d4w2c"
Sep 09 20:56:57.984582 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:57.984474    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-99rvq"]
Sep 09 20:56:57.987138 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:56:57.987115    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05fffb82_92b8_4d04_a29c_fead867763fa.slice/crio-66bfea5cd5838af0bb4bc033048455da53830efe0accabb5d7678ea2401a98dc WatchSource:0}: Error finding container 66bfea5cd5838af0bb4bc033048455da53830efe0accabb5d7678ea2401a98dc: Status 404 returned error can't find the container with id 66bfea5cd5838af0bb4bc033048455da53830efe0accabb5d7678ea2401a98dc
Sep 09 20:56:58.155959 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:58.155892    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-d4w2c"]
Sep 09 20:56:58.158083 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:56:58.158055    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8adf056_1c9d_4dd0_9428_4771e32c43ee.slice/crio-115f84b46c2620af6a556ed1e3bb5460847594aad0085afbff94f88876645b8f WatchSource:0}: Error finding container 115f84b46c2620af6a556ed1e3bb5460847594aad0085afbff94f88876645b8f: Status 404 returned error can't find the container with id 115f84b46c2620af6a556ed1e3bb5460847594aad0085afbff94f88876645b8f
Sep 09 20:56:58.922367 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:58.922332    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-99rvq" event={"ID":"05fffb82-92b8-4d04-a29c-fead867763fa","Type":"ContainerStarted","Data":"66bfea5cd5838af0bb4bc033048455da53830efe0accabb5d7678ea2401a98dc"}
Sep 09 20:56:58.923525 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:56:58.923499    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-d4w2c" event={"ID":"b8adf056-1c9d-4dd0-9428-4771e32c43ee","Type":"ContainerStarted","Data":"115f84b46c2620af6a556ed1e3bb5460847594aad0085afbff94f88876645b8f"}
Sep 09 20:57:02.942259 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:57:02.942219    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-99rvq" event={"ID":"05fffb82-92b8-4d04-a29c-fead867763fa","Type":"ContainerStarted","Data":"73883aff09699fb1e849985024ec209fd0a9de26291bdef189b75a2560525f12"}
Sep 09 20:57:02.942770 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:57:02.942320    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-579c8958f7-99rvq"
Sep 09 20:57:02.943692 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:57:02.943669    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-d4w2c" event={"ID":"b8adf056-1c9d-4dd0-9428-4771e32c43ee","Type":"ContainerStarted","Data":"b0ba2984bdab1a96d809fb0caf4fc43f5b897ca317607a2f5dc1c3f7fa22a583"}
Sep 09 20:57:02.943817 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:57:02.943802    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-6c4d864b86-d4w2c"
Sep 09 20:57:02.964490 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:57:02.964438    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-579c8958f7-99rvq" podStartSLOduration=2.863730112 podStartE2EDuration="6.964423525s" totalImagesPullingTime="4.100693413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:57:02.961531467 +0000 UTC m=+274.632161166" watchObservedRunningTime="2026-09-09 20:57:02.964423525 +0000 UTC m=+274.635053225"
Sep 09 20:57:02.983989 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:57:02.983932    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-6c4d864b86-d4w2c" podStartSLOduration=2.064437254 podStartE2EDuration="5.983914219s" totalImagesPullingTime="3.919476965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:57:02.982846888 +0000 UTC m=+274.653476586" watchObservedRunningTime="2026-09-09 20:57:02.983914219 +0000 UTC m=+274.654543918"
Sep 09 20:57:23.949484 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:57:23.949454    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-579c8958f7-99rvq"
Sep 09 20:57:23.949907 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:57:23.949695    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-6c4d864b86-d4w2c"
Sep 09 20:57:28.667074 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:57:28.667046    2585 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 20:58:18.513338 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.513295    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-644949856f-pj8s5"]
Sep 09 20:58:18.515286 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.515266    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-644949856f-pj8s5"
Sep 09 20:58:18.520787 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.520760    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 09 20:58:18.520922 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.520766    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-njth6\""
Sep 09 20:58:18.520922 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.520770    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 09 20:58:18.520922 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.520767    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Sep 09 20:58:18.539762 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.539731    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-644949856f-pj8s5"]
Sep 09 20:58:18.559578 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.559548    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-kw4nd"]
Sep 09 20:58:18.561556 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.561540    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-kw4nd"
Sep 09 20:58:18.564976 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.564956    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-xh647\""
Sep 09 20:58:18.565170 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.565156    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Sep 09 20:58:18.577759 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.577734    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-kw4nd"]
Sep 09 20:58:18.614320 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.614291    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ff1febf7-4e22-4442-a4ef-9ddb04ede5be-data\") pod \"seaweedfs-64568bcd49-kw4nd\" (UID: \"ff1febf7-4e22-4442-a4ef-9ddb04ede5be\") " pod="kserve/seaweedfs-64568bcd49-kw4nd"
Sep 09 20:58:18.614458 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.614330    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2vlx\" (UniqueName: \"kubernetes.io/projected/5860e5d0-d06a-4347-8324-aee8dc616eb0-kube-api-access-g2vlx\") pod \"kserve-controller-manager-644949856f-pj8s5\" (UID: \"5860e5d0-d06a-4347-8324-aee8dc616eb0\") " pod="kserve/kserve-controller-manager-644949856f-pj8s5"
Sep 09 20:58:18.614458 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.614420    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5860e5d0-d06a-4347-8324-aee8dc616eb0-cert\") pod \"kserve-controller-manager-644949856f-pj8s5\" (UID: \"5860e5d0-d06a-4347-8324-aee8dc616eb0\") " pod="kserve/kserve-controller-manager-644949856f-pj8s5"
Sep 09 20:58:18.614458 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.614445    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj4vg\" (UniqueName: \"kubernetes.io/projected/ff1febf7-4e22-4442-a4ef-9ddb04ede5be-kube-api-access-mj4vg\") pod \"seaweedfs-64568bcd49-kw4nd\" (UID: \"ff1febf7-4e22-4442-a4ef-9ddb04ede5be\") " pod="kserve/seaweedfs-64568bcd49-kw4nd"
Sep 09 20:58:18.714943 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.714902    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5860e5d0-d06a-4347-8324-aee8dc616eb0-cert\") pod \"kserve-controller-manager-644949856f-pj8s5\" (UID: \"5860e5d0-d06a-4347-8324-aee8dc616eb0\") " pod="kserve/kserve-controller-manager-644949856f-pj8s5"
Sep 09 20:58:18.715108 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.714949    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj4vg\" (UniqueName: \"kubernetes.io/projected/ff1febf7-4e22-4442-a4ef-9ddb04ede5be-kube-api-access-mj4vg\") pod \"seaweedfs-64568bcd49-kw4nd\" (UID: \"ff1febf7-4e22-4442-a4ef-9ddb04ede5be\") " pod="kserve/seaweedfs-64568bcd49-kw4nd"
Sep 09 20:58:18.715108 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.714992    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ff1febf7-4e22-4442-a4ef-9ddb04ede5be-data\") pod \"seaweedfs-64568bcd49-kw4nd\" (UID: \"ff1febf7-4e22-4442-a4ef-9ddb04ede5be\") " pod="kserve/seaweedfs-64568bcd49-kw4nd"
Sep 09 20:58:18.715108 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.715014    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2vlx\" (UniqueName: \"kubernetes.io/projected/5860e5d0-d06a-4347-8324-aee8dc616eb0-kube-api-access-g2vlx\") pod \"kserve-controller-manager-644949856f-pj8s5\" (UID: \"5860e5d0-d06a-4347-8324-aee8dc616eb0\") " pod="kserve/kserve-controller-manager-644949856f-pj8s5"
Sep 09 20:58:18.715457 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.715433    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ff1febf7-4e22-4442-a4ef-9ddb04ede5be-data\") pod \"seaweedfs-64568bcd49-kw4nd\" (UID: \"ff1febf7-4e22-4442-a4ef-9ddb04ede5be\") " pod="kserve/seaweedfs-64568bcd49-kw4nd"
Sep 09 20:58:18.717398 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.717379    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5860e5d0-d06a-4347-8324-aee8dc616eb0-cert\") pod \"kserve-controller-manager-644949856f-pj8s5\" (UID: \"5860e5d0-d06a-4347-8324-aee8dc616eb0\") " pod="kserve/kserve-controller-manager-644949856f-pj8s5"
Sep 09 20:58:18.735251 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.735214    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2vlx\" (UniqueName: \"kubernetes.io/projected/5860e5d0-d06a-4347-8324-aee8dc616eb0-kube-api-access-g2vlx\") pod \"kserve-controller-manager-644949856f-pj8s5\" (UID: \"5860e5d0-d06a-4347-8324-aee8dc616eb0\") " pod="kserve/kserve-controller-manager-644949856f-pj8s5"
Sep 09 20:58:18.736317 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.736284    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj4vg\" (UniqueName: \"kubernetes.io/projected/ff1febf7-4e22-4442-a4ef-9ddb04ede5be-kube-api-access-mj4vg\") pod \"seaweedfs-64568bcd49-kw4nd\" (UID: \"ff1febf7-4e22-4442-a4ef-9ddb04ede5be\") " pod="kserve/seaweedfs-64568bcd49-kw4nd"
Sep 09 20:58:18.825313 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.825275    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-644949856f-pj8s5"
Sep 09 20:58:18.870312 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:18.870288    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-kw4nd"
Sep 09 20:58:19.096440 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:19.096415    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-644949856f-pj8s5"]
Sep 09 20:58:19.098860 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:58:19.098829    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5860e5d0_d06a_4347_8324_aee8dc616eb0.slice/crio-41213fbf4967cef16495a885976223b30937cccbfee84d5c89c6e2cb0cf21f7e WatchSource:0}: Error finding container 41213fbf4967cef16495a885976223b30937cccbfee84d5c89c6e2cb0cf21f7e: Status 404 returned error can't find the container with id 41213fbf4967cef16495a885976223b30937cccbfee84d5c89c6e2cb0cf21f7e
Sep 09 20:58:19.100256 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:19.100240    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 20:58:19.151504 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:19.151476    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-kw4nd"]
Sep 09 20:58:19.153986 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:58:19.153953    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff1febf7_4e22_4442_a4ef_9ddb04ede5be.slice/crio-66227d55322661f42f55b44189b41039eca065d315f12825d9fae3e1fd4be704 WatchSource:0}: Error finding container 66227d55322661f42f55b44189b41039eca065d315f12825d9fae3e1fd4be704: Status 404 returned error can't find the container with id 66227d55322661f42f55b44189b41039eca065d315f12825d9fae3e1fd4be704
Sep 09 20:58:19.211712 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:19.211671    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-644949856f-pj8s5" event={"ID":"5860e5d0-d06a-4347-8324-aee8dc616eb0","Type":"ContainerStarted","Data":"41213fbf4967cef16495a885976223b30937cccbfee84d5c89c6e2cb0cf21f7e"}
Sep 09 20:58:19.212730 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:19.212707    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-kw4nd" event={"ID":"ff1febf7-4e22-4442-a4ef-9ddb04ede5be","Type":"ContainerStarted","Data":"66227d55322661f42f55b44189b41039eca065d315f12825d9fae3e1fd4be704"}
Sep 09 20:58:23.232850 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:23.232757    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-644949856f-pj8s5" event={"ID":"5860e5d0-d06a-4347-8324-aee8dc616eb0","Type":"ContainerStarted","Data":"2dfd69f126571307c35ed981d30eee4c70c1117f039f4146580a77b171da40bc"}
Sep 09 20:58:23.233322 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:23.232849    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-644949856f-pj8s5"
Sep 09 20:58:23.234366 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:23.234342    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-kw4nd" event={"ID":"ff1febf7-4e22-4442-a4ef-9ddb04ede5be","Type":"ContainerStarted","Data":"47d82dd36e8005ad4174a46788df69f898da0f373cd5e335f6422bdb9dda9981"}
Sep 09 20:58:23.234495 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:23.234385    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-kw4nd"
Sep 09 20:58:23.254804 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:23.254718    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-644949856f-pj8s5" podStartSLOduration=1.503894191 podStartE2EDuration="5.254696954s" totalImagesPullingTime="3.750802763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:58:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:58:23.253590766 +0000 UTC m=+354.924220480" watchObservedRunningTime="2026-09-09 20:58:23.254696954 +0000 UTC m=+354.925326655"
Sep 09 20:58:23.273769 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:23.273713    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-kw4nd" podStartSLOduration=1.485706236 podStartE2EDuration="5.273695653s" totalImagesPullingTime="3.787989417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:58:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:58:23.27292312 +0000 UTC m=+354.943552821" watchObservedRunningTime="2026-09-09 20:58:23.273695653 +0000 UTC m=+354.944325352"
Sep 09 20:58:29.240812 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:29.240775    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-kw4nd"
Sep 09 20:58:47.892471 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:47.892425    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-554c4ccdcf-2sgl8"]
Sep 09 20:58:54.243481 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:58:54.243448    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-644949856f-pj8s5"
Sep 09 20:59:02.475111 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.475061    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-644949856f-pj8s5"]
Sep 09 20:59:02.475533 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.475277    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-644949856f-pj8s5" podUID="5860e5d0-d06a-4347-8324-aee8dc616eb0" containerName="manager" containerID="cri-o://2dfd69f126571307c35ed981d30eee4c70c1117f039f4146580a77b171da40bc" gracePeriod=10
Sep 09 20:59:02.501983 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.501948    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-644949856f-8wqzq"]
Sep 09 20:59:02.503972 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.503955    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-644949856f-8wqzq"
Sep 09 20:59:02.518102 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.518070    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-644949856f-8wqzq"]
Sep 09 20:59:02.565251 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.565063    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqfgt\" (UniqueName: \"kubernetes.io/projected/478dd089-0d75-441b-be7a-015b3b91af57-kube-api-access-tqfgt\") pod \"kserve-controller-manager-644949856f-8wqzq\" (UID: \"478dd089-0d75-441b-be7a-015b3b91af57\") " pod="kserve/kserve-controller-manager-644949856f-8wqzq"
Sep 09 20:59:02.565251 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.565102    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/478dd089-0d75-441b-be7a-015b3b91af57-cert\") pod \"kserve-controller-manager-644949856f-8wqzq\" (UID: \"478dd089-0d75-441b-be7a-015b3b91af57\") " pod="kserve/kserve-controller-manager-644949856f-8wqzq"
Sep 09 20:59:02.665572 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.665527    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqfgt\" (UniqueName: \"kubernetes.io/projected/478dd089-0d75-441b-be7a-015b3b91af57-kube-api-access-tqfgt\") pod \"kserve-controller-manager-644949856f-8wqzq\" (UID: \"478dd089-0d75-441b-be7a-015b3b91af57\") " pod="kserve/kserve-controller-manager-644949856f-8wqzq"
Sep 09 20:59:02.665572 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.665569    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/478dd089-0d75-441b-be7a-015b3b91af57-cert\") pod \"kserve-controller-manager-644949856f-8wqzq\" (UID: \"478dd089-0d75-441b-be7a-015b3b91af57\") " pod="kserve/kserve-controller-manager-644949856f-8wqzq"
Sep 09 20:59:02.667978 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.667943    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/478dd089-0d75-441b-be7a-015b3b91af57-cert\") pod \"kserve-controller-manager-644949856f-8wqzq\" (UID: \"478dd089-0d75-441b-be7a-015b3b91af57\") " pod="kserve/kserve-controller-manager-644949856f-8wqzq"
Sep 09 20:59:02.675054 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.675027    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqfgt\" (UniqueName: \"kubernetes.io/projected/478dd089-0d75-441b-be7a-015b3b91af57-kube-api-access-tqfgt\") pod \"kserve-controller-manager-644949856f-8wqzq\" (UID: \"478dd089-0d75-441b-be7a-015b3b91af57\") " pod="kserve/kserve-controller-manager-644949856f-8wqzq"
Sep 09 20:59:02.712525 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.712503    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-644949856f-pj8s5"
Sep 09 20:59:02.766379 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.766285    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5860e5d0-d06a-4347-8324-aee8dc616eb0-cert\") pod \"5860e5d0-d06a-4347-8324-aee8dc616eb0\" (UID: \"5860e5d0-d06a-4347-8324-aee8dc616eb0\") "
Sep 09 20:59:02.766545 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.766393    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2vlx\" (UniqueName: \"kubernetes.io/projected/5860e5d0-d06a-4347-8324-aee8dc616eb0-kube-api-access-g2vlx\") pod \"5860e5d0-d06a-4347-8324-aee8dc616eb0\" (UID: \"5860e5d0-d06a-4347-8324-aee8dc616eb0\") "
Sep 09 20:59:02.768549 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.768515    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5860e5d0-d06a-4347-8324-aee8dc616eb0-cert" (OuterVolumeSpecName: "cert") pod "5860e5d0-d06a-4347-8324-aee8dc616eb0" (UID: "5860e5d0-d06a-4347-8324-aee8dc616eb0"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:59:02.768693 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.768552    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5860e5d0-d06a-4347-8324-aee8dc616eb0-kube-api-access-g2vlx" (OuterVolumeSpecName: "kube-api-access-g2vlx") pod "5860e5d0-d06a-4347-8324-aee8dc616eb0" (UID: "5860e5d0-d06a-4347-8324-aee8dc616eb0"). InnerVolumeSpecName "kube-api-access-g2vlx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:02.867775 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.867736    2585 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5860e5d0-d06a-4347-8324-aee8dc616eb0-cert\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:59:02.867775 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.867774    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2vlx\" (UniqueName: \"kubernetes.io/projected/5860e5d0-d06a-4347-8324-aee8dc616eb0-kube-api-access-g2vlx\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:59:02.868131 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:02.868113    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-644949856f-8wqzq"
Sep 09 20:59:03.140822 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:03.140793    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-644949856f-8wqzq"]
Sep 09 20:59:03.143238 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:59:03.143208    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod478dd089_0d75_441b_be7a_015b3b91af57.slice/crio-9d9d8e61a8862f0d0b173f02b654d63e82633517efda25211d26ecd210169041 WatchSource:0}: Error finding container 9d9d8e61a8862f0d0b173f02b654d63e82633517efda25211d26ecd210169041: Status 404 returned error can't find the container with id 9d9d8e61a8862f0d0b173f02b654d63e82633517efda25211d26ecd210169041
Sep 09 20:59:03.380630 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:03.380596    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-644949856f-8wqzq" event={"ID":"478dd089-0d75-441b-be7a-015b3b91af57","Type":"ContainerStarted","Data":"9d9d8e61a8862f0d0b173f02b654d63e82633517efda25211d26ecd210169041"}
Sep 09 20:59:03.381835 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:03.381804    2585 generic.go:356] "Generic (PLEG): container finished" podID="5860e5d0-d06a-4347-8324-aee8dc616eb0" containerID="2dfd69f126571307c35ed981d30eee4c70c1117f039f4146580a77b171da40bc" exitCode=0
Sep 09 20:59:03.381965 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:03.381860    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-644949856f-pj8s5"
Sep 09 20:59:03.381965 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:03.381910    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-644949856f-pj8s5" event={"ID":"5860e5d0-d06a-4347-8324-aee8dc616eb0","Type":"ContainerDied","Data":"2dfd69f126571307c35ed981d30eee4c70c1117f039f4146580a77b171da40bc"}
Sep 09 20:59:03.381965 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:03.381952    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-644949856f-pj8s5" event={"ID":"5860e5d0-d06a-4347-8324-aee8dc616eb0","Type":"ContainerDied","Data":"41213fbf4967cef16495a885976223b30937cccbfee84d5c89c6e2cb0cf21f7e"}
Sep 09 20:59:03.382069 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:03.381974    2585 scope.go:117] "RemoveContainer" containerID="2dfd69f126571307c35ed981d30eee4c70c1117f039f4146580a77b171da40bc"
Sep 09 20:59:03.389856 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:03.389832    2585 scope.go:117] "RemoveContainer" containerID="2dfd69f126571307c35ed981d30eee4c70c1117f039f4146580a77b171da40bc"
Sep 09 20:59:03.390179 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:59:03.390160    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dfd69f126571307c35ed981d30eee4c70c1117f039f4146580a77b171da40bc\": container with ID starting with 2dfd69f126571307c35ed981d30eee4c70c1117f039f4146580a77b171da40bc not found: ID does not exist" containerID="2dfd69f126571307c35ed981d30eee4c70c1117f039f4146580a77b171da40bc"
Sep 09 20:59:03.390236 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:03.390187    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dfd69f126571307c35ed981d30eee4c70c1117f039f4146580a77b171da40bc"} err="failed to get container status \"2dfd69f126571307c35ed981d30eee4c70c1117f039f4146580a77b171da40bc\": rpc error: code = NotFound desc = could not find container \"2dfd69f126571307c35ed981d30eee4c70c1117f039f4146580a77b171da40bc\": container with ID starting with 2dfd69f126571307c35ed981d30eee4c70c1117f039f4146580a77b171da40bc not found: ID does not exist"
Sep 09 20:59:03.402400 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:03.402331    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-644949856f-pj8s5"]
Sep 09 20:59:03.406910 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:03.406873    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-644949856f-pj8s5"]
Sep 09 20:59:04.387659 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:04.387620    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-644949856f-8wqzq" event={"ID":"478dd089-0d75-441b-be7a-015b3b91af57","Type":"ContainerStarted","Data":"11f9c67a72301e3822f0db28762980c7d58484fc7661870229debfd136a1c812"}
Sep 09 20:59:04.388162 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:04.387736    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-644949856f-8wqzq"
Sep 09 20:59:04.406611 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:04.406547    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-644949856f-8wqzq" podStartSLOduration=1.988863173 podStartE2EDuration="2.406529447s" totalImagesPullingTime="417.666274ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:59:04.405811408 +0000 UTC m=+396.076441107" watchObservedRunningTime="2026-09-09 20:59:04.406529447 +0000 UTC m=+396.077159203"
Sep 09 20:59:04.798153 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:04.798123    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5860e5d0-d06a-4347-8324-aee8dc616eb0" path="/var/lib/kubelet/pods/5860e5d0-d06a-4347-8324-aee8dc616eb0/volumes"
Sep 09 20:59:12.912105 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:12.912054    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-554c4ccdcf-2sgl8" podUID="4c48d054-b99d-4d09-b731-07260dea1100" containerName="console" containerID="cri-o://73bb0cc9d44b4f1ed60b2c243aa111159f04ddbca174dd6337835bab6f06a903" gracePeriod=15
Sep 09 20:59:13.154116 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.154089    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:59:13.243692 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.243606    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktgp4\" (UniqueName: \"kubernetes.io/projected/4c48d054-b99d-4d09-b731-07260dea1100-kube-api-access-ktgp4\") pod \"4c48d054-b99d-4d09-b731-07260dea1100\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") "
Sep 09 20:59:13.243692 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.243653    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-console-config\") pod \"4c48d054-b99d-4d09-b731-07260dea1100\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") "
Sep 09 20:59:13.243692 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.243680    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-service-ca\") pod \"4c48d054-b99d-4d09-b731-07260dea1100\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") "
Sep 09 20:59:13.243981 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.243726    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c48d054-b99d-4d09-b731-07260dea1100-console-serving-cert\") pod \"4c48d054-b99d-4d09-b731-07260dea1100\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") "
Sep 09 20:59:13.243981 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.243758    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-oauth-serving-cert\") pod \"4c48d054-b99d-4d09-b731-07260dea1100\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") "
Sep 09 20:59:13.243981 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.243785    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-trusted-ca-bundle\") pod \"4c48d054-b99d-4d09-b731-07260dea1100\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") "
Sep 09 20:59:13.243981 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.243831    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c48d054-b99d-4d09-b731-07260dea1100-console-oauth-config\") pod \"4c48d054-b99d-4d09-b731-07260dea1100\" (UID: \"4c48d054-b99d-4d09-b731-07260dea1100\") "
Sep 09 20:59:13.244174 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.244141    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-console-config" (OuterVolumeSpecName: "console-config") pod "4c48d054-b99d-4d09-b731-07260dea1100" (UID: "4c48d054-b99d-4d09-b731-07260dea1100"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:59:13.244230 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.244193    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-service-ca" (OuterVolumeSpecName: "service-ca") pod "4c48d054-b99d-4d09-b731-07260dea1100" (UID: "4c48d054-b99d-4d09-b731-07260dea1100"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:59:13.244569 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.244541    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4c48d054-b99d-4d09-b731-07260dea1100" (UID: "4c48d054-b99d-4d09-b731-07260dea1100"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:59:13.244721 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.244602    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4c48d054-b99d-4d09-b731-07260dea1100" (UID: "4c48d054-b99d-4d09-b731-07260dea1100"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:59:13.246033 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.246009    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c48d054-b99d-4d09-b731-07260dea1100-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4c48d054-b99d-4d09-b731-07260dea1100" (UID: "4c48d054-b99d-4d09-b731-07260dea1100"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:59:13.246318 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.246300    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c48d054-b99d-4d09-b731-07260dea1100-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4c48d054-b99d-4d09-b731-07260dea1100" (UID: "4c48d054-b99d-4d09-b731-07260dea1100"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:59:13.246375 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.246324    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c48d054-b99d-4d09-b731-07260dea1100-kube-api-access-ktgp4" (OuterVolumeSpecName: "kube-api-access-ktgp4") pod "4c48d054-b99d-4d09-b731-07260dea1100" (UID: "4c48d054-b99d-4d09-b731-07260dea1100"). InnerVolumeSpecName "kube-api-access-ktgp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:13.345372 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.345328    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktgp4\" (UniqueName: \"kubernetes.io/projected/4c48d054-b99d-4d09-b731-07260dea1100-kube-api-access-ktgp4\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:59:13.345372 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.345363    2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-console-config\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:59:13.345372 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.345372    2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-service-ca\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:59:13.345372 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.345380    2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c48d054-b99d-4d09-b731-07260dea1100-console-serving-cert\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:59:13.345638 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.345389    2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-oauth-serving-cert\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:59:13.345638 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.345398    2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c48d054-b99d-4d09-b731-07260dea1100-trusted-ca-bundle\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:59:13.345638 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.345407    2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c48d054-b99d-4d09-b731-07260dea1100-console-oauth-config\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 20:59:13.421633 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.421597    2585 generic.go:356] "Generic (PLEG): container finished" podID="4c48d054-b99d-4d09-b731-07260dea1100" containerID="73bb0cc9d44b4f1ed60b2c243aa111159f04ddbca174dd6337835bab6f06a903" exitCode=0
Sep 09 20:59:13.421781 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.421647    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554c4ccdcf-2sgl8"
Sep 09 20:59:13.421781 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.421689    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554c4ccdcf-2sgl8" event={"ID":"4c48d054-b99d-4d09-b731-07260dea1100","Type":"ContainerDied","Data":"73bb0cc9d44b4f1ed60b2c243aa111159f04ddbca174dd6337835bab6f06a903"}
Sep 09 20:59:13.421781 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.421743    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554c4ccdcf-2sgl8" event={"ID":"4c48d054-b99d-4d09-b731-07260dea1100","Type":"ContainerDied","Data":"4d77facc7d2a17ac54480bfe503fd89c0750a5385db951215c18d97ca087eef2"}
Sep 09 20:59:13.421781 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.421764    2585 scope.go:117] "RemoveContainer" containerID="73bb0cc9d44b4f1ed60b2c243aa111159f04ddbca174dd6337835bab6f06a903"
Sep 09 20:59:13.430551 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.430535    2585 scope.go:117] "RemoveContainer" containerID="73bb0cc9d44b4f1ed60b2c243aa111159f04ddbca174dd6337835bab6f06a903"
Sep 09 20:59:13.430858 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:59:13.430839    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73bb0cc9d44b4f1ed60b2c243aa111159f04ddbca174dd6337835bab6f06a903\": container with ID starting with 73bb0cc9d44b4f1ed60b2c243aa111159f04ddbca174dd6337835bab6f06a903 not found: ID does not exist" containerID="73bb0cc9d44b4f1ed60b2c243aa111159f04ddbca174dd6337835bab6f06a903"
Sep 09 20:59:13.430931 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.430865    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73bb0cc9d44b4f1ed60b2c243aa111159f04ddbca174dd6337835bab6f06a903"} err="failed to get container status \"73bb0cc9d44b4f1ed60b2c243aa111159f04ddbca174dd6337835bab6f06a903\": rpc error: code = NotFound desc = could not find container \"73bb0cc9d44b4f1ed60b2c243aa111159f04ddbca174dd6337835bab6f06a903\": container with ID starting with 73bb0cc9d44b4f1ed60b2c243aa111159f04ddbca174dd6337835bab6f06a903 not found: ID does not exist"
Sep 09 20:59:13.443927 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.443900    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-554c4ccdcf-2sgl8"]
Sep 09 20:59:13.447456 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:13.447431    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-554c4ccdcf-2sgl8"]
Sep 09 20:59:14.798325 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:14.798287    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c48d054-b99d-4d09-b731-07260dea1100" path="/var/lib/kubelet/pods/4c48d054-b99d-4d09-b731-07260dea1100/volumes"
Sep 09 20:59:35.397120 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:35.397079    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-644949856f-8wqzq"
Sep 09 20:59:36.688564 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.688531    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-7b954ffc95-56488"]
Sep 09 20:59:36.688968 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.688836    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5860e5d0-d06a-4347-8324-aee8dc616eb0" containerName="manager"
Sep 09 20:59:36.688968 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.688850    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5860e5d0-d06a-4347-8324-aee8dc616eb0" containerName="manager"
Sep 09 20:59:36.688968 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.688862    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c48d054-b99d-4d09-b731-07260dea1100" containerName="console"
Sep 09 20:59:36.688968 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.688867    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c48d054-b99d-4d09-b731-07260dea1100" containerName="console"
Sep 09 20:59:36.688968 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.688947    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5860e5d0-d06a-4347-8324-aee8dc616eb0" containerName="manager"
Sep 09 20:59:36.688968 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.688957    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c48d054-b99d-4d09-b731-07260dea1100" containerName="console"
Sep 09 20:59:36.693004 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.692983    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-56488"
Sep 09 20:59:36.695547 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.695523    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Sep 09 20:59:36.695632 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.695549    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-cvblf\""
Sep 09 20:59:36.704640 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.704613    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-56488"]
Sep 09 20:59:36.707681 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.707656    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7b966d7984-phpqj"]
Sep 09 20:59:36.710816 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.710798    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-phpqj"
Sep 09 20:59:36.713421 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.713395    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-rfbng\""
Sep 09 20:59:36.713690 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.713674    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\""
Sep 09 20:59:36.713746 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.713680    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Sep 09 20:59:36.723222 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.723200    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-phpqj"]
Sep 09 20:59:36.821303 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.821267    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmnfv\" (UniqueName: \"kubernetes.io/projected/beaff307-ce71-4909-95f5-ba702a791992-kube-api-access-mmnfv\") pod \"odh-model-controller-7b966d7984-phpqj\" (UID: \"beaff307-ce71-4909-95f5-ba702a791992\") " pod="kserve/odh-model-controller-7b966d7984-phpqj"
Sep 09 20:59:36.821473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.821312    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9pvh\" (UniqueName: \"kubernetes.io/projected/9e94f480-d81c-4edc-9fad-7fb1dda28645-kube-api-access-v9pvh\") pod \"model-serving-api-7b954ffc95-56488\" (UID: \"9e94f480-d81c-4edc-9fad-7fb1dda28645\") " pod="kserve/model-serving-api-7b954ffc95-56488"
Sep 09 20:59:36.821473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.821390    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/beaff307-ce71-4909-95f5-ba702a791992-metrics-certs\") pod \"odh-model-controller-7b966d7984-phpqj\" (UID: \"beaff307-ce71-4909-95f5-ba702a791992\") " pod="kserve/odh-model-controller-7b966d7984-phpqj"
Sep 09 20:59:36.821473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.821422    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e94f480-d81c-4edc-9fad-7fb1dda28645-tls-certs\") pod \"model-serving-api-7b954ffc95-56488\" (UID: \"9e94f480-d81c-4edc-9fad-7fb1dda28645\") " pod="kserve/model-serving-api-7b954ffc95-56488"
Sep 09 20:59:36.821473 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.821441    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/beaff307-ce71-4909-95f5-ba702a791992-cert\") pod \"odh-model-controller-7b966d7984-phpqj\" (UID: \"beaff307-ce71-4909-95f5-ba702a791992\") " pod="kserve/odh-model-controller-7b966d7984-phpqj"
Sep 09 20:59:36.922782 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.922750    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9pvh\" (UniqueName: \"kubernetes.io/projected/9e94f480-d81c-4edc-9fad-7fb1dda28645-kube-api-access-v9pvh\") pod \"model-serving-api-7b954ffc95-56488\" (UID: \"9e94f480-d81c-4edc-9fad-7fb1dda28645\") " pod="kserve/model-serving-api-7b954ffc95-56488"
Sep 09 20:59:36.922977 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.922808    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/beaff307-ce71-4909-95f5-ba702a791992-metrics-certs\") pod \"odh-model-controller-7b966d7984-phpqj\" (UID: \"beaff307-ce71-4909-95f5-ba702a791992\") " pod="kserve/odh-model-controller-7b966d7984-phpqj"
Sep 09 20:59:36.922977 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.922842    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e94f480-d81c-4edc-9fad-7fb1dda28645-tls-certs\") pod \"model-serving-api-7b954ffc95-56488\" (UID: \"9e94f480-d81c-4edc-9fad-7fb1dda28645\") " pod="kserve/model-serving-api-7b954ffc95-56488"
Sep 09 20:59:36.922977 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.922861    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/beaff307-ce71-4909-95f5-ba702a791992-cert\") pod \"odh-model-controller-7b966d7984-phpqj\" (UID: \"beaff307-ce71-4909-95f5-ba702a791992\") " pod="kserve/odh-model-controller-7b966d7984-phpqj"
Sep 09 20:59:36.923115 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:59:36.922980    2585 secret.go:189] Couldn't get secret kserve/odh-model-controller-metrics-tls: secret "odh-model-controller-metrics-tls" not found
Sep 09 20:59:36.923115 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.922998    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmnfv\" (UniqueName: \"kubernetes.io/projected/beaff307-ce71-4909-95f5-ba702a791992-kube-api-access-mmnfv\") pod \"odh-model-controller-7b966d7984-phpqj\" (UID: \"beaff307-ce71-4909-95f5-ba702a791992\") " pod="kserve/odh-model-controller-7b966d7984-phpqj"
Sep 09 20:59:36.923115 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:59:36.923047    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/beaff307-ce71-4909-95f5-ba702a791992-metrics-certs podName:beaff307-ce71-4909-95f5-ba702a791992 nodeName:}" failed. No retries permitted until 2026-09-09 20:59:37.423025131 +0000 UTC m=+429.093654837 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/beaff307-ce71-4909-95f5-ba702a791992-metrics-certs") pod "odh-model-controller-7b966d7984-phpqj" (UID: "beaff307-ce71-4909-95f5-ba702a791992") : secret "odh-model-controller-metrics-tls" not found
Sep 09 20:59:36.923115 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:59:36.922980    2585 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 09 20:59:36.923283 ip-10-0-129-24 kubenswrapper[2585]: E0909 20:59:36.923144    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e94f480-d81c-4edc-9fad-7fb1dda28645-tls-certs podName:9e94f480-d81c-4edc-9fad-7fb1dda28645 nodeName:}" failed. No retries permitted until 2026-09-09 20:59:37.423122082 +0000 UTC m=+429.093751779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9e94f480-d81c-4edc-9fad-7fb1dda28645-tls-certs") pod "model-serving-api-7b954ffc95-56488" (UID: "9e94f480-d81c-4edc-9fad-7fb1dda28645") : secret "model-serving-api-tls" not found
Sep 09 20:59:36.925338 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.925320    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/beaff307-ce71-4909-95f5-ba702a791992-cert\") pod \"odh-model-controller-7b966d7984-phpqj\" (UID: \"beaff307-ce71-4909-95f5-ba702a791992\") " pod="kserve/odh-model-controller-7b966d7984-phpqj"
Sep 09 20:59:36.936606 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.936577    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmnfv\" (UniqueName: \"kubernetes.io/projected/beaff307-ce71-4909-95f5-ba702a791992-kube-api-access-mmnfv\") pod \"odh-model-controller-7b966d7984-phpqj\" (UID: \"beaff307-ce71-4909-95f5-ba702a791992\") " pod="kserve/odh-model-controller-7b966d7984-phpqj"
Sep 09 20:59:36.937156 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:36.937140    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9pvh\" (UniqueName: \"kubernetes.io/projected/9e94f480-d81c-4edc-9fad-7fb1dda28645-kube-api-access-v9pvh\") pod \"model-serving-api-7b954ffc95-56488\" (UID: \"9e94f480-d81c-4edc-9fad-7fb1dda28645\") " pod="kserve/model-serving-api-7b954ffc95-56488"
Sep 09 20:59:37.427162 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:37.427118    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/beaff307-ce71-4909-95f5-ba702a791992-metrics-certs\") pod \"odh-model-controller-7b966d7984-phpqj\" (UID: \"beaff307-ce71-4909-95f5-ba702a791992\") " pod="kserve/odh-model-controller-7b966d7984-phpqj"
Sep 09 20:59:37.427162 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:37.427168    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e94f480-d81c-4edc-9fad-7fb1dda28645-tls-certs\") pod \"model-serving-api-7b954ffc95-56488\" (UID: \"9e94f480-d81c-4edc-9fad-7fb1dda28645\") " pod="kserve/model-serving-api-7b954ffc95-56488"
Sep 09 20:59:37.429576 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:37.429551    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e94f480-d81c-4edc-9fad-7fb1dda28645-tls-certs\") pod \"model-serving-api-7b954ffc95-56488\" (UID: \"9e94f480-d81c-4edc-9fad-7fb1dda28645\") " pod="kserve/model-serving-api-7b954ffc95-56488"
Sep 09 20:59:37.429710 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:37.429560    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/beaff307-ce71-4909-95f5-ba702a791992-metrics-certs\") pod \"odh-model-controller-7b966d7984-phpqj\" (UID: \"beaff307-ce71-4909-95f5-ba702a791992\") " pod="kserve/odh-model-controller-7b966d7984-phpqj"
Sep 09 20:59:37.603410 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:37.603367    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-56488"
Sep 09 20:59:37.620561 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:37.620531    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-phpqj"
Sep 09 20:59:37.887699 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:37.887642    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-56488"]
Sep 09 20:59:37.891452 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:59:37.891430    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e94f480_d81c_4edc_9fad_7fb1dda28645.slice/crio-e965ab6aa787371b5e9e9d81f018d2b02cb0be9c5b5f9e133c9778e8f3f37685 WatchSource:0}: Error finding container e965ab6aa787371b5e9e9d81f018d2b02cb0be9c5b5f9e133c9778e8f3f37685: Status 404 returned error can't find the container with id e965ab6aa787371b5e9e9d81f018d2b02cb0be9c5b5f9e133c9778e8f3f37685
Sep 09 20:59:37.914649 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:37.914510    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-phpqj"]
Sep 09 20:59:37.916943 ip-10-0-129-24 kubenswrapper[2585]: W0909 20:59:37.916920    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeaff307_ce71_4909_95f5_ba702a791992.slice/crio-0fec8d8abc3f8dbf52f2ec5920a9546cfb2e0b817a4e9457d8a0c0cc26e04241 WatchSource:0}: Error finding container 0fec8d8abc3f8dbf52f2ec5920a9546cfb2e0b817a4e9457d8a0c0cc26e04241: Status 404 returned error can't find the container with id 0fec8d8abc3f8dbf52f2ec5920a9546cfb2e0b817a4e9457d8a0c0cc26e04241
Sep 09 20:59:38.517660 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:38.517539    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-phpqj" event={"ID":"beaff307-ce71-4909-95f5-ba702a791992","Type":"ContainerStarted","Data":"0fec8d8abc3f8dbf52f2ec5920a9546cfb2e0b817a4e9457d8a0c0cc26e04241"}
Sep 09 20:59:38.520173 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:38.520141    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-56488" event={"ID":"9e94f480-d81c-4edc-9fad-7fb1dda28645","Type":"ContainerStarted","Data":"e965ab6aa787371b5e9e9d81f018d2b02cb0be9c5b5f9e133c9778e8f3f37685"}
Sep 09 20:59:40.531015 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:40.530961    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-56488" event={"ID":"9e94f480-d81c-4edc-9fad-7fb1dda28645","Type":"ContainerStarted","Data":"944c139d355c31d82a0ad5d91e98865bcf229bde5ed15a98a39b8badf9781c95"}
Sep 09 20:59:40.531447 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:40.531144    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-7b954ffc95-56488"
Sep 09 20:59:41.536440 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:41.536405    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-phpqj" event={"ID":"beaff307-ce71-4909-95f5-ba702a791992","Type":"ContainerStarted","Data":"47f36d1aac2b213d6ffc45835769c834c5b89ca91e8a8ce0c045c8a2ec66564e"}
Sep 09 20:59:41.536810 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:41.536571    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7b966d7984-phpqj"
Sep 09 20:59:41.556532 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:41.556474    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-7b966d7984-phpqj" podStartSLOduration=2.963778667 podStartE2EDuration="5.556453512s" totalImagesPullingTime="2.592674845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:59:41.554750486 +0000 UTC m=+433.225380186" watchObservedRunningTime="2026-09-09 20:59:41.556453512 +0000 UTC m=+433.227083217"
Sep 09 20:59:41.556818 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:41.556793    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-7b954ffc95-56488" podStartSLOduration=2.98646841 podStartE2EDuration="5.55678604s" totalImagesPullingTime="2.57031763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:59:40.561089585 +0000 UTC m=+432.231719285" watchObservedRunningTime="2026-09-09 20:59:41.55678604 +0000 UTC m=+433.227415741"
Sep 09 20:59:51.540632 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:51.540559    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-7b954ffc95-56488"
Sep 09 20:59:52.543499 ip-10-0-129-24 kubenswrapper[2585]: I0909 20:59:52.543469    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7b966d7984-phpqj"
Sep 09 21:01:24.376979 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.376895    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"]
Sep 09 21:01:24.380004 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.379986    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.384294 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.384265    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 09 21:01:24.384450 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.384316    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\""
Sep 09 21:01:24.384550 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.384535    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\""
Sep 09 21:01:24.384640 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.384623    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 09 21:01:24.384695 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.384660    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-c7qlq\""
Sep 09 21:01:24.385165 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.385150    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 09 21:01:24.385475 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.385448    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 09 21:01:24.397481 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.397450    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"]
Sep 09 21:01:24.561629 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.561594    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6796ff8bff-jh926\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.561803 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.561646    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6796ff8bff-jh926\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.561803 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.561769    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6796ff8bff-jh926\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.561913 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.561809    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6796ff8bff-jh926\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.561961 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.561930    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrcb6\" (UniqueName: \"kubernetes.io/projected/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kube-api-access-nrcb6\") pod \"isvc-sklearn-graph-1-predictor-6796ff8bff-jh926\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.663061 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.662975    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6796ff8bff-jh926\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.663061 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.663011    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6796ff8bff-jh926\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.663061 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.663057    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrcb6\" (UniqueName: \"kubernetes.io/projected/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kube-api-access-nrcb6\") pod \"isvc-sklearn-graph-1-predictor-6796ff8bff-jh926\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.663335 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.663079    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6796ff8bff-jh926\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.663335 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.663100    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6796ff8bff-jh926\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.663482 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.663449    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6796ff8bff-jh926\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.663725 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.663705    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6796ff8bff-jh926\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.663831 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.663812    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6796ff8bff-jh926\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.665965 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.665944    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6796ff8bff-jh926\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.672676 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.672638    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrcb6\" (UniqueName: \"kubernetes.io/projected/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kube-api-access-nrcb6\") pod \"isvc-sklearn-graph-1-predictor-6796ff8bff-jh926\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.690381 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.690354    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:24.713810 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.713781    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"]
Sep 09 21:01:24.717266 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.717247    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:24.725446 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.725418    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\""
Sep 09 21:01:24.725594 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.725501    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\""
Sep 09 21:01:24.736365 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.736333    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"]
Sep 09 21:01:24.864772 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.864736    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:24.864772 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.864780    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:24.865053 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.864811    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:24.865053 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.864944    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:24.865053 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.865002    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn9gr\" (UniqueName: \"kubernetes.io/projected/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kube-api-access-sn9gr\") pod \"isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:24.965809 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.965725    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:24.965809 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.965776    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:24.966559 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.965823    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:24.966559 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.966131    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:24.966559 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.966210    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn9gr\" (UniqueName: \"kubernetes.io/projected/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kube-api-access-sn9gr\") pod \"isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:24.966559 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.966454    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:24.967080 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:01:24.967055    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-serving-cert: secret "isvc-sklearn-graph-2-predictor-serving-cert" not found
Sep 09 21:01:24.967176 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:01:24.967163    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-proxy-tls podName:4441833d-f3d9-41ec-99c1-6b8fbec9fb59 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:25.467141732 +0000 UTC m=+537.137771416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-proxy-tls") pod "isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" (UID: "4441833d-f3d9-41ec-99c1-6b8fbec9fb59") : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found
Sep 09 21:01:24.967747 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.967721    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:24.967842 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.967754    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:24.969270 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.969248    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"]
Sep 09 21:01:24.971588 ip-10-0-129-24 kubenswrapper[2585]: W0909 21:01:24.971557    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ebc210e_07d4_477d_ba6e_5d625fa6c3d3.slice/crio-84fbb1ee286dcea111b8315eeafad18f7dd0ffb9efdec531b880c7556ad4a933 WatchSource:0}: Error finding container 84fbb1ee286dcea111b8315eeafad18f7dd0ffb9efdec531b880c7556ad4a933: Status 404 returned error can't find the container with id 84fbb1ee286dcea111b8315eeafad18f7dd0ffb9efdec531b880c7556ad4a933
Sep 09 21:01:24.976766 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:24.976709    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn9gr\" (UniqueName: \"kubernetes.io/projected/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kube-api-access-sn9gr\") pod \"isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:25.470988 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:25.470937    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:25.473462 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:25.473433    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:25.627851 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:25.627808    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:25.927488 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:25.927441    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" event={"ID":"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3","Type":"ContainerStarted","Data":"84fbb1ee286dcea111b8315eeafad18f7dd0ffb9efdec531b880c7556ad4a933"}
Sep 09 21:01:25.933458 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:25.933427    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"]
Sep 09 21:01:25.938574 ip-10-0-129-24 kubenswrapper[2585]: W0909 21:01:25.938541    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4441833d_f3d9_41ec_99c1_6b8fbec9fb59.slice/crio-7ed4e78d6072ad30b1b3b9b72cf71ed49253e9e8ffebc09d23b4bd32fb27bbfe WatchSource:0}: Error finding container 7ed4e78d6072ad30b1b3b9b72cf71ed49253e9e8ffebc09d23b4bd32fb27bbfe: Status 404 returned error can't find the container with id 7ed4e78d6072ad30b1b3b9b72cf71ed49253e9e8ffebc09d23b4bd32fb27bbfe
Sep 09 21:01:26.933287 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:26.933248    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" event={"ID":"4441833d-f3d9-41ec-99c1-6b8fbec9fb59","Type":"ContainerStarted","Data":"7ed4e78d6072ad30b1b3b9b72cf71ed49253e9e8ffebc09d23b4bd32fb27bbfe"}
Sep 09 21:01:28.945242 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:28.945204    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" event={"ID":"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3","Type":"ContainerStarted","Data":"de830af7c08e5d0c3b0cd9e936d4aa8e8dda0618898834d44ae77fb7d43750b6"}
Sep 09 21:01:28.946779 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:28.946754    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" event={"ID":"4441833d-f3d9-41ec-99c1-6b8fbec9fb59","Type":"ContainerStarted","Data":"c9bafda262a757696465701f119608d8d17433b946d54bcedd4a757f8f7a2646"}
Sep 09 21:01:32.963222 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:32.963185    2585 generic.go:356] "Generic (PLEG): container finished" podID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerID="de830af7c08e5d0c3b0cd9e936d4aa8e8dda0618898834d44ae77fb7d43750b6" exitCode=0
Sep 09 21:01:32.963709 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:32.963266    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" event={"ID":"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3","Type":"ContainerDied","Data":"de830af7c08e5d0c3b0cd9e936d4aa8e8dda0618898834d44ae77fb7d43750b6"}
Sep 09 21:01:32.965034 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:32.964969    2585 generic.go:356] "Generic (PLEG): container finished" podID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerID="c9bafda262a757696465701f119608d8d17433b946d54bcedd4a757f8f7a2646" exitCode=0
Sep 09 21:01:32.965034 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:32.965003    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" event={"ID":"4441833d-f3d9-41ec-99c1-6b8fbec9fb59","Type":"ContainerDied","Data":"c9bafda262a757696465701f119608d8d17433b946d54bcedd4a757f8f7a2646"}
Sep 09 21:01:50.070621 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:50.070561    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" event={"ID":"4441833d-f3d9-41ec-99c1-6b8fbec9fb59","Type":"ContainerStarted","Data":"082c87fa943fa7114e6104385f2e2d381537ae7cb130d252158d585fdfbb8d69"}
Sep 09 21:01:50.074468 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:50.074430    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" event={"ID":"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3","Type":"ContainerStarted","Data":"2b3e33fb66e80a0ecea351a2a76119829476654037d56e970a74cb99c3918bdc"}
Sep 09 21:01:53.089130 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:53.089086    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" event={"ID":"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3","Type":"ContainerStarted","Data":"1f3d6e6463ddbfb1b5bdda6511c94c5c17159a6b0b05a8b94dbba547a8a5eeee"}
Sep 09 21:01:53.089613 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:53.089171    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:53.089613 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:53.089208    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:53.090709 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:53.090670    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 09 21:01:53.091419 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:53.091400    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" event={"ID":"4441833d-f3d9-41ec-99c1-6b8fbec9fb59","Type":"ContainerStarted","Data":"e5a3251a1ce52648b3f1b0d08a861903b3281d3b4a19c67c9a27cf7db6d6deb9"}
Sep 09 21:01:53.091577 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:53.091558    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:53.111758 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:53.111663    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" podStartSLOduration=2.977962466 podStartE2EDuration="29.111644239s" totalImagesPullingTime="22.27801127s" totalInitContainerRuntime="3.855670503s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:53.110898872 +0000 UTC m=+564.781528567" watchObservedRunningTime="2026-09-09 21:01:53.111644239 +0000 UTC m=+564.782273939"
Sep 09 21:01:53.135447 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:53.135391    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" podStartSLOduration=4.044594269 podStartE2EDuration="29.135375034s" totalImagesPullingTime="21.310559706s" totalInitContainerRuntime="3.780221059s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:53.133796166 +0000 UTC m=+564.804425867" watchObservedRunningTime="2026-09-09 21:01:53.135375034 +0000 UTC m=+564.806004734"
Sep 09 21:01:54.095195 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:54.095141    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 09 21:01:54.095195 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:54.095188    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:01:54.096264 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:54.096242    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Sep 09 21:01:55.098782 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:55.098738    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Sep 09 21:01:59.100396 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:59.100364    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:01:59.100924 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:01:59.100864    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 09 21:02:00.103802 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:00.103771    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:02:00.104414 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:00.104383    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Sep 09 21:02:09.101814 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:09.101775    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 09 21:02:10.104345 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:10.104302    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Sep 09 21:02:19.101389 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:19.101350    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 09 21:02:20.105086 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:20.105049    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Sep 09 21:02:29.100792 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:29.100754    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 09 21:02:30.104364 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:30.104320    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Sep 09 21:02:39.101207 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:39.101166    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 09 21:02:40.105121 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:40.105078    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Sep 09 21:02:43.871389 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:43.871356    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"]
Sep 09 21:02:43.873779 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:43.873760    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"
Sep 09 21:02:43.876208 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:43.876180    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-88167-serving-cert\""
Sep 09 21:02:43.876317 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:43.876260    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-88167-kube-rbac-proxy-sar-config\""
Sep 09 21:02:43.883867 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:43.883841    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"]
Sep 09 21:02:43.915721 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:43.915677    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d832436b-5355-47b0-86d4-f18b1c9134a1-proxy-tls\") pod \"switch-graph-88167-7c488c55dd-2twc8\" (UID: \"d832436b-5355-47b0-86d4-f18b1c9134a1\") " pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"
Sep 09 21:02:43.915721 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:43.915732    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d832436b-5355-47b0-86d4-f18b1c9134a1-openshift-service-ca-bundle\") pod \"switch-graph-88167-7c488c55dd-2twc8\" (UID: \"d832436b-5355-47b0-86d4-f18b1c9134a1\") " pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"
Sep 09 21:02:44.016832 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:44.016790    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d832436b-5355-47b0-86d4-f18b1c9134a1-proxy-tls\") pod \"switch-graph-88167-7c488c55dd-2twc8\" (UID: \"d832436b-5355-47b0-86d4-f18b1c9134a1\") " pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"
Sep 09 21:02:44.016832 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:44.016834    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d832436b-5355-47b0-86d4-f18b1c9134a1-openshift-service-ca-bundle\") pod \"switch-graph-88167-7c488c55dd-2twc8\" (UID: \"d832436b-5355-47b0-86d4-f18b1c9134a1\") " pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"
Sep 09 21:02:44.017063 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:02:44.016994    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-88167-serving-cert: secret "switch-graph-88167-serving-cert" not found
Sep 09 21:02:44.017106 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:02:44.017088    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d832436b-5355-47b0-86d4-f18b1c9134a1-proxy-tls podName:d832436b-5355-47b0-86d4-f18b1c9134a1 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:44.517070038 +0000 UTC m=+616.187699716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d832436b-5355-47b0-86d4-f18b1c9134a1-proxy-tls") pod "switch-graph-88167-7c488c55dd-2twc8" (UID: "d832436b-5355-47b0-86d4-f18b1c9134a1") : secret "switch-graph-88167-serving-cert" not found
Sep 09 21:02:44.017639 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:44.017622    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d832436b-5355-47b0-86d4-f18b1c9134a1-openshift-service-ca-bundle\") pod \"switch-graph-88167-7c488c55dd-2twc8\" (UID: \"d832436b-5355-47b0-86d4-f18b1c9134a1\") " pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"
Sep 09 21:02:44.520588 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:44.520541    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d832436b-5355-47b0-86d4-f18b1c9134a1-proxy-tls\") pod \"switch-graph-88167-7c488c55dd-2twc8\" (UID: \"d832436b-5355-47b0-86d4-f18b1c9134a1\") " pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"
Sep 09 21:02:44.522976 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:44.522952    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d832436b-5355-47b0-86d4-f18b1c9134a1-proxy-tls\") pod \"switch-graph-88167-7c488c55dd-2twc8\" (UID: \"d832436b-5355-47b0-86d4-f18b1c9134a1\") " pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"
Sep 09 21:02:44.784539 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:44.784504    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"
Sep 09 21:02:45.057962 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:45.057935    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"]
Sep 09 21:02:45.060358 ip-10-0-129-24 kubenswrapper[2585]: W0909 21:02:45.060328    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd832436b_5355_47b0_86d4_f18b1c9134a1.slice/crio-d8afcc780f4c406d6522cb90f63cff0f00673aa2bead8cd7c0780331a9eae6f3 WatchSource:0}: Error finding container d8afcc780f4c406d6522cb90f63cff0f00673aa2bead8cd7c0780331a9eae6f3: Status 404 returned error can't find the container with id d8afcc780f4c406d6522cb90f63cff0f00673aa2bead8cd7c0780331a9eae6f3
Sep 09 21:02:45.300580 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:45.300546    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8" event={"ID":"d832436b-5355-47b0-86d4-f18b1c9134a1","Type":"ContainerStarted","Data":"d8afcc780f4c406d6522cb90f63cff0f00673aa2bead8cd7c0780331a9eae6f3"}
Sep 09 21:02:47.311109 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:47.311077    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8" event={"ID":"d832436b-5355-47b0-86d4-f18b1c9134a1","Type":"ContainerStarted","Data":"075071793caf5da4357e274fef9feded28f54e6e2e261c1d6915f727ec5bbb39"}
Sep 09 21:02:47.311477 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:47.311260    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"
Sep 09 21:02:47.332348 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:47.332296    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8" podStartSLOduration=2.174482747 podStartE2EDuration="4.332280689s" totalImagesPullingTime="2.157797942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:47.332047305 +0000 UTC m=+619.002677005" watchObservedRunningTime="2026-09-09 21:02:47.332280689 +0000 UTC m=+619.002910439"
Sep 09 21:02:49.101013 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:49.100968    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 09 21:02:50.104424 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:50.104382    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Sep 09 21:02:53.321842 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:53.321771    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"
Sep 09 21:02:54.261058 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:54.261026    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"]
Sep 09 21:02:54.261242 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:54.261200    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8" podUID="d832436b-5355-47b0-86d4-f18b1c9134a1" containerName="switch-graph-88167" containerID="cri-o://075071793caf5da4357e274fef9feded28f54e6e2e261c1d6915f727ec5bbb39" gracePeriod=30
Sep 09 21:02:58.319357 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:58.319320    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8" podUID="d832436b-5355-47b0-86d4-f18b1c9134a1" containerName="switch-graph-88167" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:02:59.101461 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:02:59.101422    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 09 21:03:00.104465 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:00.104428    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Sep 09 21:03:01.793657 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:01.793607    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 09 21:03:03.319393 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:03.319350    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8" podUID="d832436b-5355-47b0-86d4-f18b1c9134a1" containerName="switch-graph-88167" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:03:08.319815 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:08.319778    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8" podUID="d832436b-5355-47b0-86d4-f18b1c9134a1" containerName="switch-graph-88167" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:03:08.321179 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:08.319929    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"
Sep 09 21:03:10.105095 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:10.105060    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:03:11.794728 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:11.794696    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:03:13.320072 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:13.320035    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8" podUID="d832436b-5355-47b0-86d4-f18b1c9134a1" containerName="switch-graph-88167" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:03:18.319418 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:18.319376    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8" podUID="d832436b-5355-47b0-86d4-f18b1c9134a1" containerName="switch-graph-88167" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:03:23.319943 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:23.319872    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8" podUID="d832436b-5355-47b0-86d4-f18b1c9134a1" containerName="switch-graph-88167" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:03:24.396145 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.396120    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"
Sep 09 21:03:24.460414 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.460378    2585 generic.go:356] "Generic (PLEG): container finished" podID="d832436b-5355-47b0-86d4-f18b1c9134a1" containerID="075071793caf5da4357e274fef9feded28f54e6e2e261c1d6915f727ec5bbb39" exitCode=0
Sep 09 21:03:24.460592 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.460422    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"
Sep 09 21:03:24.460592 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.460420    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8" event={"ID":"d832436b-5355-47b0-86d4-f18b1c9134a1","Type":"ContainerDied","Data":"075071793caf5da4357e274fef9feded28f54e6e2e261c1d6915f727ec5bbb39"}
Sep 09 21:03:24.460592 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.460538    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8" event={"ID":"d832436b-5355-47b0-86d4-f18b1c9134a1","Type":"ContainerDied","Data":"d8afcc780f4c406d6522cb90f63cff0f00673aa2bead8cd7c0780331a9eae6f3"}
Sep 09 21:03:24.460592 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.460561    2585 scope.go:117] "RemoveContainer" containerID="075071793caf5da4357e274fef9feded28f54e6e2e261c1d6915f727ec5bbb39"
Sep 09 21:03:24.468415 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.468393    2585 scope.go:117] "RemoveContainer" containerID="075071793caf5da4357e274fef9feded28f54e6e2e261c1d6915f727ec5bbb39"
Sep 09 21:03:24.468725 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:03:24.468706    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"075071793caf5da4357e274fef9feded28f54e6e2e261c1d6915f727ec5bbb39\": container with ID starting with 075071793caf5da4357e274fef9feded28f54e6e2e261c1d6915f727ec5bbb39 not found: ID does not exist" containerID="075071793caf5da4357e274fef9feded28f54e6e2e261c1d6915f727ec5bbb39"
Sep 09 21:03:24.468784 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.468735    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"075071793caf5da4357e274fef9feded28f54e6e2e261c1d6915f727ec5bbb39"} err="failed to get container status \"075071793caf5da4357e274fef9feded28f54e6e2e261c1d6915f727ec5bbb39\": rpc error: code = NotFound desc = could not find container \"075071793caf5da4357e274fef9feded28f54e6e2e261c1d6915f727ec5bbb39\": container with ID starting with 075071793caf5da4357e274fef9feded28f54e6e2e261c1d6915f727ec5bbb39 not found: ID does not exist"
Sep 09 21:03:24.561981 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.561947    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d832436b-5355-47b0-86d4-f18b1c9134a1-openshift-service-ca-bundle\") pod \"d832436b-5355-47b0-86d4-f18b1c9134a1\" (UID: \"d832436b-5355-47b0-86d4-f18b1c9134a1\") "
Sep 09 21:03:24.562170 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.562009    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d832436b-5355-47b0-86d4-f18b1c9134a1-proxy-tls\") pod \"d832436b-5355-47b0-86d4-f18b1c9134a1\" (UID: \"d832436b-5355-47b0-86d4-f18b1c9134a1\") "
Sep 09 21:03:24.562321 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.562298    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d832436b-5355-47b0-86d4-f18b1c9134a1-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "d832436b-5355-47b0-86d4-f18b1c9134a1" (UID: "d832436b-5355-47b0-86d4-f18b1c9134a1"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:03:24.564126 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.564108    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d832436b-5355-47b0-86d4-f18b1c9134a1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d832436b-5355-47b0-86d4-f18b1c9134a1" (UID: "d832436b-5355-47b0-86d4-f18b1c9134a1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:03:24.662704 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.662643    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d832436b-5355-47b0-86d4-f18b1c9134a1-openshift-service-ca-bundle\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:03:24.662704 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.662696    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d832436b-5355-47b0-86d4-f18b1c9134a1-proxy-tls\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:03:24.782159 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.782122    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"]
Sep 09 21:03:24.790324 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.790284    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-88167-7c488c55dd-2twc8"]
Sep 09 21:03:24.798444 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:24.798413    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d832436b-5355-47b0-86d4-f18b1c9134a1" path="/var/lib/kubelet/pods/d832436b-5355-47b0-86d4-f18b1c9134a1/volumes"
Sep 09 21:03:33.901569 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:33.901535    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"]
Sep 09 21:03:33.902055 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:33.901861    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d832436b-5355-47b0-86d4-f18b1c9134a1" containerName="switch-graph-88167"
Sep 09 21:03:33.902055 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:33.901874    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d832436b-5355-47b0-86d4-f18b1c9134a1" containerName="switch-graph-88167"
Sep 09 21:03:33.902055 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:33.901947    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d832436b-5355-47b0-86d4-f18b1c9134a1" containerName="switch-graph-88167"
Sep 09 21:03:33.904272 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:33.904254    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"
Sep 09 21:03:33.907022 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:33.906996    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\""
Sep 09 21:03:33.907271 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:33.907253    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\""
Sep 09 21:03:33.915153 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:33.915130    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"]
Sep 09 21:03:33.937043 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:33.937011    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d53c364e-76da-4e9c-8df7-f48cfbcdd483-openshift-service-ca-bundle\") pod \"model-chainer-7fc46d5b45-sndvm\" (UID: \"d53c364e-76da-4e9c-8df7-f48cfbcdd483\") " pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"
Sep 09 21:03:33.937203 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:33.937051    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d53c364e-76da-4e9c-8df7-f48cfbcdd483-proxy-tls\") pod \"model-chainer-7fc46d5b45-sndvm\" (UID: \"d53c364e-76da-4e9c-8df7-f48cfbcdd483\") " pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"
Sep 09 21:03:34.038340 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:34.038293    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d53c364e-76da-4e9c-8df7-f48cfbcdd483-openshift-service-ca-bundle\") pod \"model-chainer-7fc46d5b45-sndvm\" (UID: \"d53c364e-76da-4e9c-8df7-f48cfbcdd483\") " pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"
Sep 09 21:03:34.038511 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:34.038383    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d53c364e-76da-4e9c-8df7-f48cfbcdd483-proxy-tls\") pod \"model-chainer-7fc46d5b45-sndvm\" (UID: \"d53c364e-76da-4e9c-8df7-f48cfbcdd483\") " pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"
Sep 09 21:03:34.039009 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:34.038979    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d53c364e-76da-4e9c-8df7-f48cfbcdd483-openshift-service-ca-bundle\") pod \"model-chainer-7fc46d5b45-sndvm\" (UID: \"d53c364e-76da-4e9c-8df7-f48cfbcdd483\") " pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"
Sep 09 21:03:34.040819 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:34.040799    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d53c364e-76da-4e9c-8df7-f48cfbcdd483-proxy-tls\") pod \"model-chainer-7fc46d5b45-sndvm\" (UID: \"d53c364e-76da-4e9c-8df7-f48cfbcdd483\") " pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"
Sep 09 21:03:34.214798 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:34.214692    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"
Sep 09 21:03:34.485537 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:34.485452    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"]
Sep 09 21:03:34.488561 ip-10-0-129-24 kubenswrapper[2585]: W0909 21:03:34.488536    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd53c364e_76da_4e9c_8df7_f48cfbcdd483.slice/crio-d04d647afda2a056b44ed3e7ee86d6c42d78fb2254cae85f87fa5ebc31936712 WatchSource:0}: Error finding container d04d647afda2a056b44ed3e7ee86d6c42d78fb2254cae85f87fa5ebc31936712: Status 404 returned error can't find the container with id d04d647afda2a056b44ed3e7ee86d6c42d78fb2254cae85f87fa5ebc31936712
Sep 09 21:03:34.499176 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:34.499150    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm" event={"ID":"d53c364e-76da-4e9c-8df7-f48cfbcdd483","Type":"ContainerStarted","Data":"d04d647afda2a056b44ed3e7ee86d6c42d78fb2254cae85f87fa5ebc31936712"}
Sep 09 21:03:35.504159 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:35.504117    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm" event={"ID":"d53c364e-76da-4e9c-8df7-f48cfbcdd483","Type":"ContainerStarted","Data":"07ca005c2c240ec8591d705bb5c3b254806370898d3824deb4b8b37c84924661"}
Sep 09 21:03:35.504604 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:35.504251    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"
Sep 09 21:03:41.513735 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:41.513699    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"
Sep 09 21:03:41.535438 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:41.535390    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm" podStartSLOduration=8.535374523 podStartE2EDuration="8.535374523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:03:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:35.526147455 +0000 UTC m=+667.196777149" watchObservedRunningTime="2026-09-09 21:03:41.535374523 +0000 UTC m=+673.206004222"
Sep 09 21:03:43.990141 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:43.990106    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"]
Sep 09 21:03:43.990592 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:43.990316    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm" podUID="d53c364e-76da-4e9c-8df7-f48cfbcdd483" containerName="model-chainer" containerID="cri-o://07ca005c2c240ec8591d705bb5c3b254806370898d3824deb4b8b37c84924661" gracePeriod=30
Sep 09 21:03:44.061281 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:44.061247    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"]
Sep 09 21:03:44.061518 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:44.061498    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kserve-container" containerID="cri-o://2b3e33fb66e80a0ecea351a2a76119829476654037d56e970a74cb99c3918bdc" gracePeriod=30
Sep 09 21:03:44.061637 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:44.061585    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kube-rbac-proxy" containerID="cri-o://1f3d6e6463ddbfb1b5bdda6511c94c5c17159a6b0b05a8b94dbba547a8a5eeee" gracePeriod=30
Sep 09 21:03:44.095758 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:44.095720    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Sep 09 21:03:44.183852 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:44.183822    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"]
Sep 09 21:03:44.184168 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:44.184136    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kserve-container" containerID="cri-o://082c87fa943fa7114e6104385f2e2d381537ae7cb130d252158d585fdfbb8d69" gracePeriod=30
Sep 09 21:03:44.184295 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:44.184173    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kube-rbac-proxy" containerID="cri-o://e5a3251a1ce52648b3f1b0d08a861903b3281d3b4a19c67c9a27cf7db6d6deb9" gracePeriod=30
Sep 09 21:03:44.541724 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:44.541525    2585 generic.go:356] "Generic (PLEG): container finished" podID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerID="1f3d6e6463ddbfb1b5bdda6511c94c5c17159a6b0b05a8b94dbba547a8a5eeee" exitCode=2
Sep 09 21:03:44.541724 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:44.541659    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" event={"ID":"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3","Type":"ContainerDied","Data":"1f3d6e6463ddbfb1b5bdda6511c94c5c17159a6b0b05a8b94dbba547a8a5eeee"}
Sep 09 21:03:44.544502 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:44.544470    2585 generic.go:356] "Generic (PLEG): container finished" podID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerID="e5a3251a1ce52648b3f1b0d08a861903b3281d3b4a19c67c9a27cf7db6d6deb9" exitCode=2
Sep 09 21:03:44.544622 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:44.544505    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" event={"ID":"4441833d-f3d9-41ec-99c1-6b8fbec9fb59","Type":"ContainerDied","Data":"e5a3251a1ce52648b3f1b0d08a861903b3281d3b4a19c67c9a27cf7db6d6deb9"}
Sep 09 21:03:45.099234 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:45.099189    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Sep 09 21:03:46.512851 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:46.512812    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm" podUID="d53c364e-76da-4e9c-8df7-f48cfbcdd483" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:03:49.002815 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.002789    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:03:49.065743 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.065710    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kserve-provision-location\") pod \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") "
Sep 09 21:03:49.065937 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.065770    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrcb6\" (UniqueName: \"kubernetes.io/projected/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kube-api-access-nrcb6\") pod \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") "
Sep 09 21:03:49.065937 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.065797    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-cabundle-cert\") pod \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") "
Sep 09 21:03:49.065937 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.065840    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kube-rbac-proxy-sar-config\") pod \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") "
Sep 09 21:03:49.065937 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.065898    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-proxy-tls\") pod \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\" (UID: \"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3\") "
Sep 09 21:03:49.066213 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.066153    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" (UID: "6ebc210e-07d4-477d-ba6e-5d625fa6c3d3"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:49.066213 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.066180    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" (UID: "6ebc210e-07d4-477d-ba6e-5d625fa6c3d3"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:03:49.066213 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.066162    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" (UID: "6ebc210e-07d4-477d-ba6e-5d625fa6c3d3"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:03:49.068027 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.068003    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" (UID: "6ebc210e-07d4-477d-ba6e-5d625fa6c3d3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:03:49.068136 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.068112    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kube-api-access-nrcb6" (OuterVolumeSpecName: "kube-api-access-nrcb6") pod "6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" (UID: "6ebc210e-07d4-477d-ba6e-5d625fa6c3d3"). InnerVolumeSpecName "kube-api-access-nrcb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:03:49.167635 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.167597    2585 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kserve-provision-location\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:03:49.167635 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.167637    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrcb6\" (UniqueName: \"kubernetes.io/projected/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kube-api-access-nrcb6\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:03:49.167858 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.167652    2585 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-cabundle-cert\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:03:49.167858 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.167668    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:03:49.167858 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.167681    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3-proxy-tls\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:03:49.222733 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.222708    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:03:49.268192 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.268160    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kserve-provision-location\") pod \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") "
Sep 09 21:03:49.268373 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.268215    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kube-rbac-proxy-sar-config\") pod \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") "
Sep 09 21:03:49.268373 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.268248    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-proxy-tls\") pod \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") "
Sep 09 21:03:49.268373 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.268265    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sn9gr\" (UniqueName: \"kubernetes.io/projected/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kube-api-access-sn9gr\") pod \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") "
Sep 09 21:03:49.268373 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.268295    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-cabundle-cert\") pod \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\" (UID: \"4441833d-f3d9-41ec-99c1-6b8fbec9fb59\") "
Sep 09 21:03:49.268593 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.268528    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4441833d-f3d9-41ec-99c1-6b8fbec9fb59" (UID: "4441833d-f3d9-41ec-99c1-6b8fbec9fb59"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:03:49.268694 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.268657    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4441833d-f3d9-41ec-99c1-6b8fbec9fb59" (UID: "4441833d-f3d9-41ec-99c1-6b8fbec9fb59"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:03:49.268819 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.268722    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "4441833d-f3d9-41ec-99c1-6b8fbec9fb59" (UID: "4441833d-f3d9-41ec-99c1-6b8fbec9fb59"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:03:49.270420 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.270395    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kube-api-access-sn9gr" (OuterVolumeSpecName: "kube-api-access-sn9gr") pod "4441833d-f3d9-41ec-99c1-6b8fbec9fb59" (UID: "4441833d-f3d9-41ec-99c1-6b8fbec9fb59"). InnerVolumeSpecName "kube-api-access-sn9gr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:03:49.270495 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.270474    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4441833d-f3d9-41ec-99c1-6b8fbec9fb59" (UID: "4441833d-f3d9-41ec-99c1-6b8fbec9fb59"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:03:49.369111 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.369013    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-proxy-tls\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:03:49.369111 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.369049    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn9gr\" (UniqueName: \"kubernetes.io/projected/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kube-api-access-sn9gr\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:03:49.369111 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.369060    2585 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-cabundle-cert\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:03:49.369111 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.369069    2585 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kserve-provision-location\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:03:49.369111 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.369079    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4441833d-f3d9-41ec-99c1-6b8fbec9fb59-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:03:49.566427 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.566381    2585 generic.go:356] "Generic (PLEG): container finished" podID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerID="082c87fa943fa7114e6104385f2e2d381537ae7cb130d252158d585fdfbb8d69" exitCode=0
Sep 09 21:03:49.566575 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.566448    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"
Sep 09 21:03:49.566575 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.566447    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" event={"ID":"4441833d-f3d9-41ec-99c1-6b8fbec9fb59","Type":"ContainerDied","Data":"082c87fa943fa7114e6104385f2e2d381537ae7cb130d252158d585fdfbb8d69"}
Sep 09 21:03:49.566575 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.566521    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg" event={"ID":"4441833d-f3d9-41ec-99c1-6b8fbec9fb59","Type":"ContainerDied","Data":"7ed4e78d6072ad30b1b3b9b72cf71ed49253e9e8ffebc09d23b4bd32fb27bbfe"}
Sep 09 21:03:49.566575 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.566562    2585 scope.go:117] "RemoveContainer" containerID="e5a3251a1ce52648b3f1b0d08a861903b3281d3b4a19c67c9a27cf7db6d6deb9"
Sep 09 21:03:49.568945 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.568864    2585 generic.go:356] "Generic (PLEG): container finished" podID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerID="2b3e33fb66e80a0ecea351a2a76119829476654037d56e970a74cb99c3918bdc" exitCode=0
Sep 09 21:03:49.568945 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.568913    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" event={"ID":"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3","Type":"ContainerDied","Data":"2b3e33fb66e80a0ecea351a2a76119829476654037d56e970a74cb99c3918bdc"}
Sep 09 21:03:49.568945 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.568932    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"
Sep 09 21:03:49.569109 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.568979    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926" event={"ID":"6ebc210e-07d4-477d-ba6e-5d625fa6c3d3","Type":"ContainerDied","Data":"84fbb1ee286dcea111b8315eeafad18f7dd0ffb9efdec531b880c7556ad4a933"}
Sep 09 21:03:49.576303 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.576284    2585 scope.go:117] "RemoveContainer" containerID="082c87fa943fa7114e6104385f2e2d381537ae7cb130d252158d585fdfbb8d69"
Sep 09 21:03:49.583771 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.583752    2585 scope.go:117] "RemoveContainer" containerID="c9bafda262a757696465701f119608d8d17433b946d54bcedd4a757f8f7a2646"
Sep 09 21:03:49.591409 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.591385    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"]
Sep 09 21:03:49.591571 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.591556    2585 scope.go:117] "RemoveContainer" containerID="e5a3251a1ce52648b3f1b0d08a861903b3281d3b4a19c67c9a27cf7db6d6deb9"
Sep 09 21:03:49.591857 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:03:49.591835    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5a3251a1ce52648b3f1b0d08a861903b3281d3b4a19c67c9a27cf7db6d6deb9\": container with ID starting with e5a3251a1ce52648b3f1b0d08a861903b3281d3b4a19c67c9a27cf7db6d6deb9 not found: ID does not exist" containerID="e5a3251a1ce52648b3f1b0d08a861903b3281d3b4a19c67c9a27cf7db6d6deb9"
Sep 09 21:03:49.591947 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.591870    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5a3251a1ce52648b3f1b0d08a861903b3281d3b4a19c67c9a27cf7db6d6deb9"} err="failed to get container status \"e5a3251a1ce52648b3f1b0d08a861903b3281d3b4a19c67c9a27cf7db6d6deb9\": rpc error: code = NotFound desc = could not find container \"e5a3251a1ce52648b3f1b0d08a861903b3281d3b4a19c67c9a27cf7db6d6deb9\": container with ID starting with e5a3251a1ce52648b3f1b0d08a861903b3281d3b4a19c67c9a27cf7db6d6deb9 not found: ID does not exist"
Sep 09 21:03:49.591947 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.591906    2585 scope.go:117] "RemoveContainer" containerID="082c87fa943fa7114e6104385f2e2d381537ae7cb130d252158d585fdfbb8d69"
Sep 09 21:03:49.592163 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:03:49.592145    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"082c87fa943fa7114e6104385f2e2d381537ae7cb130d252158d585fdfbb8d69\": container with ID starting with 082c87fa943fa7114e6104385f2e2d381537ae7cb130d252158d585fdfbb8d69 not found: ID does not exist" containerID="082c87fa943fa7114e6104385f2e2d381537ae7cb130d252158d585fdfbb8d69"
Sep 09 21:03:49.592204 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.592169    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"082c87fa943fa7114e6104385f2e2d381537ae7cb130d252158d585fdfbb8d69"} err="failed to get container status \"082c87fa943fa7114e6104385f2e2d381537ae7cb130d252158d585fdfbb8d69\": rpc error: code = NotFound desc = could not find container \"082c87fa943fa7114e6104385f2e2d381537ae7cb130d252158d585fdfbb8d69\": container with ID starting with 082c87fa943fa7114e6104385f2e2d381537ae7cb130d252158d585fdfbb8d69 not found: ID does not exist"
Sep 09 21:03:49.592204 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.592185    2585 scope.go:117] "RemoveContainer" containerID="c9bafda262a757696465701f119608d8d17433b946d54bcedd4a757f8f7a2646"
Sep 09 21:03:49.592471 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:03:49.592451    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9bafda262a757696465701f119608d8d17433b946d54bcedd4a757f8f7a2646\": container with ID starting with c9bafda262a757696465701f119608d8d17433b946d54bcedd4a757f8f7a2646 not found: ID does not exist" containerID="c9bafda262a757696465701f119608d8d17433b946d54bcedd4a757f8f7a2646"
Sep 09 21:03:49.592527 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.592476    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9bafda262a757696465701f119608d8d17433b946d54bcedd4a757f8f7a2646"} err="failed to get container status \"c9bafda262a757696465701f119608d8d17433b946d54bcedd4a757f8f7a2646\": rpc error: code = NotFound desc = could not find container \"c9bafda262a757696465701f119608d8d17433b946d54bcedd4a757f8f7a2646\": container with ID starting with c9bafda262a757696465701f119608d8d17433b946d54bcedd4a757f8f7a2646 not found: ID does not exist"
Sep 09 21:03:49.592527 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.592493    2585 scope.go:117] "RemoveContainer" containerID="1f3d6e6463ddbfb1b5bdda6511c94c5c17159a6b0b05a8b94dbba547a8a5eeee"
Sep 09 21:03:49.595904 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.595870    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-fd6f5569-nwxpg"]
Sep 09 21:03:49.599764 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.599748    2585 scope.go:117] "RemoveContainer" containerID="2b3e33fb66e80a0ecea351a2a76119829476654037d56e970a74cb99c3918bdc"
Sep 09 21:03:49.606913 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.606893    2585 scope.go:117] "RemoveContainer" containerID="de830af7c08e5d0c3b0cd9e936d4aa8e8dda0618898834d44ae77fb7d43750b6"
Sep 09 21:03:49.607987 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.607961    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"]
Sep 09 21:03:49.614555 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.614532    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6796ff8bff-jh926"]
Sep 09 21:03:49.614729 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.614715    2585 scope.go:117] "RemoveContainer" containerID="1f3d6e6463ddbfb1b5bdda6511c94c5c17159a6b0b05a8b94dbba547a8a5eeee"
Sep 09 21:03:49.615051 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:03:49.615033    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f3d6e6463ddbfb1b5bdda6511c94c5c17159a6b0b05a8b94dbba547a8a5eeee\": container with ID starting with 1f3d6e6463ddbfb1b5bdda6511c94c5c17159a6b0b05a8b94dbba547a8a5eeee not found: ID does not exist" containerID="1f3d6e6463ddbfb1b5bdda6511c94c5c17159a6b0b05a8b94dbba547a8a5eeee"
Sep 09 21:03:49.615096 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.615058    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f3d6e6463ddbfb1b5bdda6511c94c5c17159a6b0b05a8b94dbba547a8a5eeee"} err="failed to get container status \"1f3d6e6463ddbfb1b5bdda6511c94c5c17159a6b0b05a8b94dbba547a8a5eeee\": rpc error: code = NotFound desc = could not find container \"1f3d6e6463ddbfb1b5bdda6511c94c5c17159a6b0b05a8b94dbba547a8a5eeee\": container with ID starting with 1f3d6e6463ddbfb1b5bdda6511c94c5c17159a6b0b05a8b94dbba547a8a5eeee not found: ID does not exist"
Sep 09 21:03:49.615096 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.615075    2585 scope.go:117] "RemoveContainer" containerID="2b3e33fb66e80a0ecea351a2a76119829476654037d56e970a74cb99c3918bdc"
Sep 09 21:03:49.615383 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:03:49.615365    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b3e33fb66e80a0ecea351a2a76119829476654037d56e970a74cb99c3918bdc\": container with ID starting with 2b3e33fb66e80a0ecea351a2a76119829476654037d56e970a74cb99c3918bdc not found: ID does not exist" containerID="2b3e33fb66e80a0ecea351a2a76119829476654037d56e970a74cb99c3918bdc"
Sep 09 21:03:49.615446 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.615386    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b3e33fb66e80a0ecea351a2a76119829476654037d56e970a74cb99c3918bdc"} err="failed to get container status \"2b3e33fb66e80a0ecea351a2a76119829476654037d56e970a74cb99c3918bdc\": rpc error: code = NotFound desc = could not find container \"2b3e33fb66e80a0ecea351a2a76119829476654037d56e970a74cb99c3918bdc\": container with ID starting with 2b3e33fb66e80a0ecea351a2a76119829476654037d56e970a74cb99c3918bdc not found: ID does not exist"
Sep 09 21:03:49.615446 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.615401    2585 scope.go:117] "RemoveContainer" containerID="de830af7c08e5d0c3b0cd9e936d4aa8e8dda0618898834d44ae77fb7d43750b6"
Sep 09 21:03:49.615640 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:03:49.615624    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de830af7c08e5d0c3b0cd9e936d4aa8e8dda0618898834d44ae77fb7d43750b6\": container with ID starting with de830af7c08e5d0c3b0cd9e936d4aa8e8dda0618898834d44ae77fb7d43750b6 not found: ID does not exist" containerID="de830af7c08e5d0c3b0cd9e936d4aa8e8dda0618898834d44ae77fb7d43750b6"
Sep 09 21:03:49.615679 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:49.615643    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de830af7c08e5d0c3b0cd9e936d4aa8e8dda0618898834d44ae77fb7d43750b6"} err="failed to get container status \"de830af7c08e5d0c3b0cd9e936d4aa8e8dda0618898834d44ae77fb7d43750b6\": rpc error: code = NotFound desc = could not find container \"de830af7c08e5d0c3b0cd9e936d4aa8e8dda0618898834d44ae77fb7d43750b6\": container with ID starting with de830af7c08e5d0c3b0cd9e936d4aa8e8dda0618898834d44ae77fb7d43750b6 not found: ID does not exist"
Sep 09 21:03:50.798209 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:50.798179    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" path="/var/lib/kubelet/pods/4441833d-f3d9-41ec-99c1-6b8fbec9fb59/volumes"
Sep 09 21:03:50.798655 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:50.798642    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" path="/var/lib/kubelet/pods/6ebc210e-07d4-477d-ba6e-5d625fa6c3d3/volumes"
Sep 09 21:03:51.512468 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:51.512431    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm" podUID="d53c364e-76da-4e9c-8df7-f48cfbcdd483" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:03:56.512346 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:56.512307    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm" podUID="d53c364e-76da-4e9c-8df7-f48cfbcdd483" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:03:56.512799 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:03:56.512451    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"
Sep 09 21:04:01.512289 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:01.512252    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm" podUID="d53c364e-76da-4e9c-8df7-f48cfbcdd483" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:04:04.572249 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572215    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"]
Sep 09 21:04:04.572669 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572527    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="storage-initializer"
Sep 09 21:04:04.572669 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572538    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="storage-initializer"
Sep 09 21:04:04.572669 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572545    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kube-rbac-proxy"
Sep 09 21:04:04.572669 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572551    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kube-rbac-proxy"
Sep 09 21:04:04.572669 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572564    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kube-rbac-proxy"
Sep 09 21:04:04.572669 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572571    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kube-rbac-proxy"
Sep 09 21:04:04.572669 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572580    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kserve-container"
Sep 09 21:04:04.572669 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572585    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kserve-container"
Sep 09 21:04:04.572669 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572593    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="storage-initializer"
Sep 09 21:04:04.572669 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572598    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="storage-initializer"
Sep 09 21:04:04.572669 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572607    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kserve-container"
Sep 09 21:04:04.572669 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572612    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kserve-container"
Sep 09 21:04:04.572669 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572651    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kserve-container"
Sep 09 21:04:04.572669 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572658    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kube-rbac-proxy"
Sep 09 21:04:04.572669 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572665    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="4441833d-f3d9-41ec-99c1-6b8fbec9fb59" containerName="kserve-container"
Sep 09 21:04:04.572669 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.572671    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ebc210e-07d4-477d-ba6e-5d625fa6c3d3" containerName="kube-rbac-proxy"
Sep 09 21:04:04.574548 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.574526    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"
Sep 09 21:04:04.577086 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.577061    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-67ffc-kube-rbac-proxy-sar-config\""
Sep 09 21:04:04.577086 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.577061    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-67ffc-serving-cert\""
Sep 09 21:04:04.588036 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.588011    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"]
Sep 09 21:04:04.697659 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.697625    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/710951a4-39b2-4f2b-abda-c4bb63269ade-proxy-tls\") pod \"switch-graph-67ffc-7d94c49c97-vqqxb\" (UID: \"710951a4-39b2-4f2b-abda-c4bb63269ade\") " pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"
Sep 09 21:04:04.697828 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.697683    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/710951a4-39b2-4f2b-abda-c4bb63269ade-openshift-service-ca-bundle\") pod \"switch-graph-67ffc-7d94c49c97-vqqxb\" (UID: \"710951a4-39b2-4f2b-abda-c4bb63269ade\") " pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"
Sep 09 21:04:04.798215 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.798186    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/710951a4-39b2-4f2b-abda-c4bb63269ade-openshift-service-ca-bundle\") pod \"switch-graph-67ffc-7d94c49c97-vqqxb\" (UID: \"710951a4-39b2-4f2b-abda-c4bb63269ade\") " pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"
Sep 09 21:04:04.798420 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.798234    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/710951a4-39b2-4f2b-abda-c4bb63269ade-proxy-tls\") pod \"switch-graph-67ffc-7d94c49c97-vqqxb\" (UID: \"710951a4-39b2-4f2b-abda-c4bb63269ade\") " pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"
Sep 09 21:04:04.798420 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:04:04.798347    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-67ffc-serving-cert: secret "switch-graph-67ffc-serving-cert" not found
Sep 09 21:04:04.798420 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:04:04.798405    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/710951a4-39b2-4f2b-abda-c4bb63269ade-proxy-tls podName:710951a4-39b2-4f2b-abda-c4bb63269ade nodeName:}" failed. No retries permitted until 2026-09-09 21:04:05.298383913 +0000 UTC m=+696.969013590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/710951a4-39b2-4f2b-abda-c4bb63269ade-proxy-tls") pod "switch-graph-67ffc-7d94c49c97-vqqxb" (UID: "710951a4-39b2-4f2b-abda-c4bb63269ade") : secret "switch-graph-67ffc-serving-cert" not found
Sep 09 21:04:04.798945 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:04.798913    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/710951a4-39b2-4f2b-abda-c4bb63269ade-openshift-service-ca-bundle\") pod \"switch-graph-67ffc-7d94c49c97-vqqxb\" (UID: \"710951a4-39b2-4f2b-abda-c4bb63269ade\") " pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"
Sep 09 21:04:05.302750 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:05.302705    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/710951a4-39b2-4f2b-abda-c4bb63269ade-proxy-tls\") pod \"switch-graph-67ffc-7d94c49c97-vqqxb\" (UID: \"710951a4-39b2-4f2b-abda-c4bb63269ade\") " pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"
Sep 09 21:04:05.302954 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:04:05.302856    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-67ffc-serving-cert: secret "switch-graph-67ffc-serving-cert" not found
Sep 09 21:04:05.302954 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:04:05.302942    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/710951a4-39b2-4f2b-abda-c4bb63269ade-proxy-tls podName:710951a4-39b2-4f2b-abda-c4bb63269ade nodeName:}" failed. No retries permitted until 2026-09-09 21:04:06.302927196 +0000 UTC m=+697.973556873 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/710951a4-39b2-4f2b-abda-c4bb63269ade-proxy-tls") pod "switch-graph-67ffc-7d94c49c97-vqqxb" (UID: "710951a4-39b2-4f2b-abda-c4bb63269ade") : secret "switch-graph-67ffc-serving-cert" not found
Sep 09 21:04:06.312368 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:06.312329    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/710951a4-39b2-4f2b-abda-c4bb63269ade-proxy-tls\") pod \"switch-graph-67ffc-7d94c49c97-vqqxb\" (UID: \"710951a4-39b2-4f2b-abda-c4bb63269ade\") " pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"
Sep 09 21:04:06.314685 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:06.314662    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/710951a4-39b2-4f2b-abda-c4bb63269ade-proxy-tls\") pod \"switch-graph-67ffc-7d94c49c97-vqqxb\" (UID: \"710951a4-39b2-4f2b-abda-c4bb63269ade\") " pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"
Sep 09 21:04:06.385706 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:06.385652    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"
Sep 09 21:04:06.512184 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:06.512143    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm" podUID="d53c364e-76da-4e9c-8df7-f48cfbcdd483" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:04:06.673046 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:06.672967    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"]
Sep 09 21:04:06.676626 ip-10-0-129-24 kubenswrapper[2585]: W0909 21:04:06.676597    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod710951a4_39b2_4f2b_abda_c4bb63269ade.slice/crio-97fda431d73f655e70774ed90868a7cf43c04e1ad049ed28bbf6fc0f2a1efeed WatchSource:0}: Error finding container 97fda431d73f655e70774ed90868a7cf43c04e1ad049ed28bbf6fc0f2a1efeed: Status 404 returned error can't find the container with id 97fda431d73f655e70774ed90868a7cf43c04e1ad049ed28bbf6fc0f2a1efeed
Sep 09 21:04:07.640025 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:07.639992    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb" event={"ID":"710951a4-39b2-4f2b-abda-c4bb63269ade","Type":"ContainerStarted","Data":"9b008f7c0b450d180d64113fb66e45eb33cff64b805afe60a2553a3215eb3358"}
Sep 09 21:04:07.640442 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:07.640043    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb" event={"ID":"710951a4-39b2-4f2b-abda-c4bb63269ade","Type":"ContainerStarted","Data":"97fda431d73f655e70774ed90868a7cf43c04e1ad049ed28bbf6fc0f2a1efeed"}
Sep 09 21:04:07.640442 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:07.640073    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"
Sep 09 21:04:07.659304 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:07.659242    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb" podStartSLOduration=3.659224363 podStartE2EDuration="3.659224363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:04:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:04:07.658613199 +0000 UTC m=+699.329242900" watchObservedRunningTime="2026-09-09 21:04:07.659224363 +0000 UTC m=+699.329854062"
Sep 09 21:04:11.512367 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:11.512324    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm" podUID="d53c364e-76da-4e9c-8df7-f48cfbcdd483" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:04:13.649724 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:13.649690    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"
Sep 09 21:04:14.142224 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.142200    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"
Sep 09 21:04:14.276001 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.275906    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d53c364e-76da-4e9c-8df7-f48cfbcdd483-openshift-service-ca-bundle\") pod \"d53c364e-76da-4e9c-8df7-f48cfbcdd483\" (UID: \"d53c364e-76da-4e9c-8df7-f48cfbcdd483\") "
Sep 09 21:04:14.276001 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.275972    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d53c364e-76da-4e9c-8df7-f48cfbcdd483-proxy-tls\") pod \"d53c364e-76da-4e9c-8df7-f48cfbcdd483\" (UID: \"d53c364e-76da-4e9c-8df7-f48cfbcdd483\") "
Sep 09 21:04:14.276293 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.276268    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d53c364e-76da-4e9c-8df7-f48cfbcdd483-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "d53c364e-76da-4e9c-8df7-f48cfbcdd483" (UID: "d53c364e-76da-4e9c-8df7-f48cfbcdd483"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:04:14.278060 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.278038    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d53c364e-76da-4e9c-8df7-f48cfbcdd483-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d53c364e-76da-4e9c-8df7-f48cfbcdd483" (UID: "d53c364e-76da-4e9c-8df7-f48cfbcdd483"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:04:14.376658 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.376601    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d53c364e-76da-4e9c-8df7-f48cfbcdd483-proxy-tls\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:04:14.376658 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.376649    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d53c364e-76da-4e9c-8df7-f48cfbcdd483-openshift-service-ca-bundle\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:04:14.668062 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.668025    2585 generic.go:356] "Generic (PLEG): container finished" podID="d53c364e-76da-4e9c-8df7-f48cfbcdd483" containerID="07ca005c2c240ec8591d705bb5c3b254806370898d3824deb4b8b37c84924661" exitCode=0
Sep 09 21:04:14.668473 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.668072    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"
Sep 09 21:04:14.668473 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.668113    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm" event={"ID":"d53c364e-76da-4e9c-8df7-f48cfbcdd483","Type":"ContainerDied","Data":"07ca005c2c240ec8591d705bb5c3b254806370898d3824deb4b8b37c84924661"}
Sep 09 21:04:14.668473 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.668171    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm" event={"ID":"d53c364e-76da-4e9c-8df7-f48cfbcdd483","Type":"ContainerDied","Data":"d04d647afda2a056b44ed3e7ee86d6c42d78fb2254cae85f87fa5ebc31936712"}
Sep 09 21:04:14.668473 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.668194    2585 scope.go:117] "RemoveContainer" containerID="07ca005c2c240ec8591d705bb5c3b254806370898d3824deb4b8b37c84924661"
Sep 09 21:04:14.676752 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.676731    2585 scope.go:117] "RemoveContainer" containerID="07ca005c2c240ec8591d705bb5c3b254806370898d3824deb4b8b37c84924661"
Sep 09 21:04:14.677119 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:04:14.677051    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07ca005c2c240ec8591d705bb5c3b254806370898d3824deb4b8b37c84924661\": container with ID starting with 07ca005c2c240ec8591d705bb5c3b254806370898d3824deb4b8b37c84924661 not found: ID does not exist" containerID="07ca005c2c240ec8591d705bb5c3b254806370898d3824deb4b8b37c84924661"
Sep 09 21:04:14.677119 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.677076    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07ca005c2c240ec8591d705bb5c3b254806370898d3824deb4b8b37c84924661"} err="failed to get container status \"07ca005c2c240ec8591d705bb5c3b254806370898d3824deb4b8b37c84924661\": rpc error: code = NotFound desc = could not find container \"07ca005c2c240ec8591d705bb5c3b254806370898d3824deb4b8b37c84924661\": container with ID starting with 07ca005c2c240ec8591d705bb5c3b254806370898d3824deb4b8b37c84924661 not found: ID does not exist"
Sep 09 21:04:14.694469 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.691594    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"]
Sep 09 21:04:14.697839 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.697804    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7fc46d5b45-sndvm"]
Sep 09 21:04:14.798661 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:14.798630    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d53c364e-76da-4e9c-8df7-f48cfbcdd483" path="/var/lib/kubelet/pods/d53c364e-76da-4e9c-8df7-f48cfbcdd483/volumes"
Sep 09 21:04:44.238310 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:44.238273    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"]
Sep 09 21:04:44.238709 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:44.238652    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d53c364e-76da-4e9c-8df7-f48cfbcdd483" containerName="model-chainer"
Sep 09 21:04:44.238709 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:44.238668    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d53c364e-76da-4e9c-8df7-f48cfbcdd483" containerName="model-chainer"
Sep 09 21:04:44.238804 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:44.238744    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d53c364e-76da-4e9c-8df7-f48cfbcdd483" containerName="model-chainer"
Sep 09 21:04:44.243131 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:44.243110    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"
Sep 09 21:04:44.245563 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:44.245543    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-456dd-kube-rbac-proxy-sar-config\""
Sep 09 21:04:44.245563 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:44.245556    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-456dd-serving-cert\""
Sep 09 21:04:44.250626 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:44.250606    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"]
Sep 09 21:04:44.307218 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:44.307183    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/310677ad-9164-44c5-bee9-0fcde74291bc-openshift-service-ca-bundle\") pod \"sequence-graph-456dd-bb57fc7d7-qq4p8\" (UID: \"310677ad-9164-44c5-bee9-0fcde74291bc\") " pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"
Sep 09 21:04:44.307375 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:44.307246    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/310677ad-9164-44c5-bee9-0fcde74291bc-proxy-tls\") pod \"sequence-graph-456dd-bb57fc7d7-qq4p8\" (UID: \"310677ad-9164-44c5-bee9-0fcde74291bc\") " pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"
Sep 09 21:04:44.408609 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:44.408566    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/310677ad-9164-44c5-bee9-0fcde74291bc-proxy-tls\") pod \"sequence-graph-456dd-bb57fc7d7-qq4p8\" (UID: \"310677ad-9164-44c5-bee9-0fcde74291bc\") " pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"
Sep 09 21:04:44.408786 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:44.408622    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/310677ad-9164-44c5-bee9-0fcde74291bc-openshift-service-ca-bundle\") pod \"sequence-graph-456dd-bb57fc7d7-qq4p8\" (UID: \"310677ad-9164-44c5-bee9-0fcde74291bc\") " pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"
Sep 09 21:04:44.408786 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:04:44.408729    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-456dd-serving-cert: secret "sequence-graph-456dd-serving-cert" not found
Sep 09 21:04:44.408904 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:04:44.408807    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/310677ad-9164-44c5-bee9-0fcde74291bc-proxy-tls podName:310677ad-9164-44c5-bee9-0fcde74291bc nodeName:}" failed. No retries permitted until 2026-09-09 21:04:44.908791027 +0000 UTC m=+736.579420705 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/310677ad-9164-44c5-bee9-0fcde74291bc-proxy-tls") pod "sequence-graph-456dd-bb57fc7d7-qq4p8" (UID: "310677ad-9164-44c5-bee9-0fcde74291bc") : secret "sequence-graph-456dd-serving-cert" not found
Sep 09 21:04:44.409262 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:44.409245    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/310677ad-9164-44c5-bee9-0fcde74291bc-openshift-service-ca-bundle\") pod \"sequence-graph-456dd-bb57fc7d7-qq4p8\" (UID: \"310677ad-9164-44c5-bee9-0fcde74291bc\") " pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"
Sep 09 21:04:44.912519 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:44.912473    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/310677ad-9164-44c5-bee9-0fcde74291bc-proxy-tls\") pod \"sequence-graph-456dd-bb57fc7d7-qq4p8\" (UID: \"310677ad-9164-44c5-bee9-0fcde74291bc\") " pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"
Sep 09 21:04:44.914988 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:44.914965    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/310677ad-9164-44c5-bee9-0fcde74291bc-proxy-tls\") pod \"sequence-graph-456dd-bb57fc7d7-qq4p8\" (UID: \"310677ad-9164-44c5-bee9-0fcde74291bc\") " pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"
Sep 09 21:04:45.154039 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:45.153983    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"
Sep 09 21:04:45.423474 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:45.423444    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"]
Sep 09 21:04:45.425984 ip-10-0-129-24 kubenswrapper[2585]: W0909 21:04:45.425952    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod310677ad_9164_44c5_bee9_0fcde74291bc.slice/crio-358a2d35a14b4d1b9a9dc2f80812fbf1dfd44ae3489f7ee8d2b4f48c02da967a WatchSource:0}: Error finding container 358a2d35a14b4d1b9a9dc2f80812fbf1dfd44ae3489f7ee8d2b4f48c02da967a: Status 404 returned error can't find the container with id 358a2d35a14b4d1b9a9dc2f80812fbf1dfd44ae3489f7ee8d2b4f48c02da967a
Sep 09 21:04:45.799082 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:45.799052    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8" event={"ID":"310677ad-9164-44c5-bee9-0fcde74291bc","Type":"ContainerStarted","Data":"3269253dcb07f570965a1bd13972a483eeaf23ef973ebe503e68175a9a762bb9"}
Sep 09 21:04:45.799267 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:45.799095    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8" event={"ID":"310677ad-9164-44c5-bee9-0fcde74291bc","Type":"ContainerStarted","Data":"358a2d35a14b4d1b9a9dc2f80812fbf1dfd44ae3489f7ee8d2b4f48c02da967a"}
Sep 09 21:04:45.799267 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:45.799127    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"
Sep 09 21:04:45.820565 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:45.820506    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8" podStartSLOduration=1.820492097 podStartE2EDuration="1.820492097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:04:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:04:45.818658807 +0000 UTC m=+737.489288506" watchObservedRunningTime="2026-09-09 21:04:45.820492097 +0000 UTC m=+737.491121796"
Sep 09 21:04:51.807433 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:04:51.807398    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"
Sep 09 21:12:19.220257 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:19.218602    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"]
Sep 09 21:12:19.220257 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:19.219017    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb" podUID="710951a4-39b2-4f2b-abda-c4bb63269ade" containerName="switch-graph-67ffc" containerID="cri-o://9b008f7c0b450d180d64113fb66e45eb33cff64b805afe60a2553a3215eb3358" gracePeriod=30
Sep 09 21:12:23.647647 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:23.647609    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb" podUID="710951a4-39b2-4f2b-abda-c4bb63269ade" containerName="switch-graph-67ffc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:12:28.648027 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:28.647984    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb" podUID="710951a4-39b2-4f2b-abda-c4bb63269ade" containerName="switch-graph-67ffc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:12:33.647677 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:33.647625    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb" podUID="710951a4-39b2-4f2b-abda-c4bb63269ade" containerName="switch-graph-67ffc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:12:33.648058 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:33.647730    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"
Sep 09 21:12:38.648182 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:38.648147    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb" podUID="710951a4-39b2-4f2b-abda-c4bb63269ade" containerName="switch-graph-67ffc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:12:43.648177 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:43.648137    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb" podUID="710951a4-39b2-4f2b-abda-c4bb63269ade" containerName="switch-graph-67ffc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:12:48.648501 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:48.648456    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb" podUID="710951a4-39b2-4f2b-abda-c4bb63269ade" containerName="switch-graph-67ffc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:12:49.367171 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:49.367144    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"
Sep 09 21:12:49.463967 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:49.463931    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/710951a4-39b2-4f2b-abda-c4bb63269ade-openshift-service-ca-bundle\") pod \"710951a4-39b2-4f2b-abda-c4bb63269ade\" (UID: \"710951a4-39b2-4f2b-abda-c4bb63269ade\") "
Sep 09 21:12:49.464177 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:49.464000    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/710951a4-39b2-4f2b-abda-c4bb63269ade-proxy-tls\") pod \"710951a4-39b2-4f2b-abda-c4bb63269ade\" (UID: \"710951a4-39b2-4f2b-abda-c4bb63269ade\") "
Sep 09 21:12:49.464318 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:49.464295    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/710951a4-39b2-4f2b-abda-c4bb63269ade-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "710951a4-39b2-4f2b-abda-c4bb63269ade" (UID: "710951a4-39b2-4f2b-abda-c4bb63269ade"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:12:49.466012 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:49.465991    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/710951a4-39b2-4f2b-abda-c4bb63269ade-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "710951a4-39b2-4f2b-abda-c4bb63269ade" (UID: "710951a4-39b2-4f2b-abda-c4bb63269ade"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:12:49.565439 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:49.565394    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/710951a4-39b2-4f2b-abda-c4bb63269ade-proxy-tls\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:12:49.565439 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:49.565438    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/710951a4-39b2-4f2b-abda-c4bb63269ade-openshift-service-ca-bundle\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:12:49.586632 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:49.586598    2585 generic.go:356] "Generic (PLEG): container finished" podID="710951a4-39b2-4f2b-abda-c4bb63269ade" containerID="9b008f7c0b450d180d64113fb66e45eb33cff64b805afe60a2553a3215eb3358" exitCode=0
Sep 09 21:12:49.586770 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:49.586646    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"
Sep 09 21:12:49.586770 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:49.586652    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb" event={"ID":"710951a4-39b2-4f2b-abda-c4bb63269ade","Type":"ContainerDied","Data":"9b008f7c0b450d180d64113fb66e45eb33cff64b805afe60a2553a3215eb3358"}
Sep 09 21:12:49.586770 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:49.586694    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb" event={"ID":"710951a4-39b2-4f2b-abda-c4bb63269ade","Type":"ContainerDied","Data":"97fda431d73f655e70774ed90868a7cf43c04e1ad049ed28bbf6fc0f2a1efeed"}
Sep 09 21:12:49.586770 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:49.586714    2585 scope.go:117] "RemoveContainer" containerID="9b008f7c0b450d180d64113fb66e45eb33cff64b805afe60a2553a3215eb3358"
Sep 09 21:12:49.595138 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:49.595120    2585 scope.go:117] "RemoveContainer" containerID="9b008f7c0b450d180d64113fb66e45eb33cff64b805afe60a2553a3215eb3358"
Sep 09 21:12:49.595393 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:12:49.595370    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b008f7c0b450d180d64113fb66e45eb33cff64b805afe60a2553a3215eb3358\": container with ID starting with 9b008f7c0b450d180d64113fb66e45eb33cff64b805afe60a2553a3215eb3358 not found: ID does not exist" containerID="9b008f7c0b450d180d64113fb66e45eb33cff64b805afe60a2553a3215eb3358"
Sep 09 21:12:49.595446 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:49.595401    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b008f7c0b450d180d64113fb66e45eb33cff64b805afe60a2553a3215eb3358"} err="failed to get container status \"9b008f7c0b450d180d64113fb66e45eb33cff64b805afe60a2553a3215eb3358\": rpc error: code = NotFound desc = could not find container \"9b008f7c0b450d180d64113fb66e45eb33cff64b805afe60a2553a3215eb3358\": container with ID starting with 9b008f7c0b450d180d64113fb66e45eb33cff64b805afe60a2553a3215eb3358 not found: ID does not exist"
Sep 09 21:12:49.615562 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:49.615532    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"]
Sep 09 21:12:49.618891 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:49.618851    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-67ffc-7d94c49c97-vqqxb"]
Sep 09 21:12:50.798293 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:50.798259    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="710951a4-39b2-4f2b-abda-c4bb63269ade" path="/var/lib/kubelet/pods/710951a4-39b2-4f2b-abda-c4bb63269ade/volumes"
Sep 09 21:12:58.866511 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:58.866471    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"]
Sep 09 21:12:58.867354 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:12:58.867295    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8" podUID="310677ad-9164-44c5-bee9-0fcde74291bc" containerName="sequence-graph-456dd" containerID="cri-o://3269253dcb07f570965a1bd13972a483eeaf23ef973ebe503e68175a9a762bb9" gracePeriod=30
Sep 09 21:13:01.805893 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:01.805842    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8" podUID="310677ad-9164-44c5-bee9-0fcde74291bc" containerName="sequence-graph-456dd" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:13:06.805800 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:06.805761    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8" podUID="310677ad-9164-44c5-bee9-0fcde74291bc" containerName="sequence-graph-456dd" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:13:11.806258 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:11.806221    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8" podUID="310677ad-9164-44c5-bee9-0fcde74291bc" containerName="sequence-graph-456dd" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:13:11.806720 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:11.806325    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"
Sep 09 21:13:16.806464 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:16.806428    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8" podUID="310677ad-9164-44c5-bee9-0fcde74291bc" containerName="sequence-graph-456dd" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:13:19.540253 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:19.540219    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"]
Sep 09 21:13:19.540638 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:19.540506    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="710951a4-39b2-4f2b-abda-c4bb63269ade" containerName="switch-graph-67ffc"
Sep 09 21:13:19.540638 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:19.540517    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="710951a4-39b2-4f2b-abda-c4bb63269ade" containerName="switch-graph-67ffc"
Sep 09 21:13:19.540638 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:19.540572    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="710951a4-39b2-4f2b-abda-c4bb63269ade" containerName="switch-graph-67ffc"
Sep 09 21:13:19.543480 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:19.543458    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"
Sep 09 21:13:19.546989 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:19.546956    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-71ac5-kube-rbac-proxy-sar-config\""
Sep 09 21:13:19.547084 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:19.546962    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-71ac5-serving-cert\""
Sep 09 21:13:19.563469 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:19.563443    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"]
Sep 09 21:13:19.605956 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:19.605925    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e114a8f-50ad-4b31-a44c-0a146366ee71-openshift-service-ca-bundle\") pod \"ensemble-graph-71ac5-5b87cdfb8f-d9rgb\" (UID: \"0e114a8f-50ad-4b31-a44c-0a146366ee71\") " pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"
Sep 09 21:13:19.606109 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:19.605987    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0e114a8f-50ad-4b31-a44c-0a146366ee71-proxy-tls\") pod \"ensemble-graph-71ac5-5b87cdfb8f-d9rgb\" (UID: \"0e114a8f-50ad-4b31-a44c-0a146366ee71\") " pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"
Sep 09 21:13:19.706782 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:19.706746    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0e114a8f-50ad-4b31-a44c-0a146366ee71-proxy-tls\") pod \"ensemble-graph-71ac5-5b87cdfb8f-d9rgb\" (UID: \"0e114a8f-50ad-4b31-a44c-0a146366ee71\") " pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"
Sep 09 21:13:19.706986 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:19.706803    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e114a8f-50ad-4b31-a44c-0a146366ee71-openshift-service-ca-bundle\") pod \"ensemble-graph-71ac5-5b87cdfb8f-d9rgb\" (UID: \"0e114a8f-50ad-4b31-a44c-0a146366ee71\") " pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"
Sep 09 21:13:19.706986 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:13:19.706911    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-71ac5-serving-cert: secret "ensemble-graph-71ac5-serving-cert" not found
Sep 09 21:13:19.706986 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:13:19.706979    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e114a8f-50ad-4b31-a44c-0a146366ee71-proxy-tls podName:0e114a8f-50ad-4b31-a44c-0a146366ee71 nodeName:}" failed. No retries permitted until 2026-09-09 21:13:20.206959141 +0000 UTC m=+1251.877588821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/0e114a8f-50ad-4b31-a44c-0a146366ee71-proxy-tls") pod "ensemble-graph-71ac5-5b87cdfb8f-d9rgb" (UID: "0e114a8f-50ad-4b31-a44c-0a146366ee71") : secret "ensemble-graph-71ac5-serving-cert" not found
Sep 09 21:13:19.707489 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:19.707471    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e114a8f-50ad-4b31-a44c-0a146366ee71-openshift-service-ca-bundle\") pod \"ensemble-graph-71ac5-5b87cdfb8f-d9rgb\" (UID: \"0e114a8f-50ad-4b31-a44c-0a146366ee71\") " pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"
Sep 09 21:13:20.210715 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:20.210681    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0e114a8f-50ad-4b31-a44c-0a146366ee71-proxy-tls\") pod \"ensemble-graph-71ac5-5b87cdfb8f-d9rgb\" (UID: \"0e114a8f-50ad-4b31-a44c-0a146366ee71\") " pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"
Sep 09 21:13:20.213051 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:20.213031    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0e114a8f-50ad-4b31-a44c-0a146366ee71-proxy-tls\") pod \"ensemble-graph-71ac5-5b87cdfb8f-d9rgb\" (UID: \"0e114a8f-50ad-4b31-a44c-0a146366ee71\") " pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"
Sep 09 21:13:20.453507 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:20.453469    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"
Sep 09 21:13:20.748058 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:20.747971    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"]
Sep 09 21:13:20.750866 ip-10-0-129-24 kubenswrapper[2585]: W0909 21:13:20.750836    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e114a8f_50ad_4b31_a44c_0a146366ee71.slice/crio-58768ad519a1593ae54b895f5f619a42e6e85622be6b4d873782376e1272f7de WatchSource:0}: Error finding container 58768ad519a1593ae54b895f5f619a42e6e85622be6b4d873782376e1272f7de: Status 404 returned error can't find the container with id 58768ad519a1593ae54b895f5f619a42e6e85622be6b4d873782376e1272f7de
Sep 09 21:13:21.706118 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:21.706085    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb" event={"ID":"0e114a8f-50ad-4b31-a44c-0a146366ee71","Type":"ContainerStarted","Data":"bd5db2bb87f024777482bcfcf8afeb1ab5e8f22b70d2f4bb0b1091eae64ee590"}
Sep 09 21:13:21.706358 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:21.706161    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb" event={"ID":"0e114a8f-50ad-4b31-a44c-0a146366ee71","Type":"ContainerStarted","Data":"58768ad519a1593ae54b895f5f619a42e6e85622be6b4d873782376e1272f7de"}
Sep 09 21:13:21.706358 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:21.706183    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"
Sep 09 21:13:21.731130 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:21.731081    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb" podStartSLOduration=2.731065601 podStartE2EDuration="2.731065601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:13:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:13:21.730040482 +0000 UTC m=+1253.400670195" watchObservedRunningTime="2026-09-09 21:13:21.731065601 +0000 UTC m=+1253.401695301"
Sep 09 21:13:21.806479 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:21.806439    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8" podUID="310677ad-9164-44c5-bee9-0fcde74291bc" containerName="sequence-graph-456dd" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:13:26.806703 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:26.806664    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8" podUID="310677ad-9164-44c5-bee9-0fcde74291bc" containerName="sequence-graph-456dd" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:13:27.715871 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:27.715837    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"
Sep 09 21:13:29.014975 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.014950    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"
Sep 09 21:13:29.084129 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.084052    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/310677ad-9164-44c5-bee9-0fcde74291bc-proxy-tls\") pod \"310677ad-9164-44c5-bee9-0fcde74291bc\" (UID: \"310677ad-9164-44c5-bee9-0fcde74291bc\") "
Sep 09 21:13:29.084299 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.084125    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/310677ad-9164-44c5-bee9-0fcde74291bc-openshift-service-ca-bundle\") pod \"310677ad-9164-44c5-bee9-0fcde74291bc\" (UID: \"310677ad-9164-44c5-bee9-0fcde74291bc\") "
Sep 09 21:13:29.084501 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.084475    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/310677ad-9164-44c5-bee9-0fcde74291bc-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "310677ad-9164-44c5-bee9-0fcde74291bc" (UID: "310677ad-9164-44c5-bee9-0fcde74291bc"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:13:29.086112 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.086083    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/310677ad-9164-44c5-bee9-0fcde74291bc-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "310677ad-9164-44c5-bee9-0fcde74291bc" (UID: "310677ad-9164-44c5-bee9-0fcde74291bc"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:13:29.184826 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.184786    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/310677ad-9164-44c5-bee9-0fcde74291bc-proxy-tls\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:13:29.184826 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.184819    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/310677ad-9164-44c5-bee9-0fcde74291bc-openshift-service-ca-bundle\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:13:29.678970 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.678932    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"]
Sep 09 21:13:29.679131 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.679115    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb" podUID="0e114a8f-50ad-4b31-a44c-0a146366ee71" containerName="ensemble-graph-71ac5" containerID="cri-o://bd5db2bb87f024777482bcfcf8afeb1ab5e8f22b70d2f4bb0b1091eae64ee590" gracePeriod=30
Sep 09 21:13:29.737692 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.737655    2585 generic.go:356] "Generic (PLEG): container finished" podID="310677ad-9164-44c5-bee9-0fcde74291bc" containerID="3269253dcb07f570965a1bd13972a483eeaf23ef973ebe503e68175a9a762bb9" exitCode=0
Sep 09 21:13:29.737692 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.737693    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"
Sep 09 21:13:29.737895 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.737718    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8" event={"ID":"310677ad-9164-44c5-bee9-0fcde74291bc","Type":"ContainerDied","Data":"3269253dcb07f570965a1bd13972a483eeaf23ef973ebe503e68175a9a762bb9"}
Sep 09 21:13:29.737895 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.737776    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8" event={"ID":"310677ad-9164-44c5-bee9-0fcde74291bc","Type":"ContainerDied","Data":"358a2d35a14b4d1b9a9dc2f80812fbf1dfd44ae3489f7ee8d2b4f48c02da967a"}
Sep 09 21:13:29.737895 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.737809    2585 scope.go:117] "RemoveContainer" containerID="3269253dcb07f570965a1bd13972a483eeaf23ef973ebe503e68175a9a762bb9"
Sep 09 21:13:29.746782 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.746762    2585 scope.go:117] "RemoveContainer" containerID="3269253dcb07f570965a1bd13972a483eeaf23ef973ebe503e68175a9a762bb9"
Sep 09 21:13:29.747123 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:13:29.747102    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3269253dcb07f570965a1bd13972a483eeaf23ef973ebe503e68175a9a762bb9\": container with ID starting with 3269253dcb07f570965a1bd13972a483eeaf23ef973ebe503e68175a9a762bb9 not found: ID does not exist" containerID="3269253dcb07f570965a1bd13972a483eeaf23ef973ebe503e68175a9a762bb9"
Sep 09 21:13:29.747224 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.747130    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3269253dcb07f570965a1bd13972a483eeaf23ef973ebe503e68175a9a762bb9"} err="failed to get container status \"3269253dcb07f570965a1bd13972a483eeaf23ef973ebe503e68175a9a762bb9\": rpc error: code = NotFound desc = could not find container \"3269253dcb07f570965a1bd13972a483eeaf23ef973ebe503e68175a9a762bb9\": container with ID starting with 3269253dcb07f570965a1bd13972a483eeaf23ef973ebe503e68175a9a762bb9 not found: ID does not exist"
Sep 09 21:13:29.777717 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.777687    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"]
Sep 09 21:13:29.783292 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:29.783265    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-456dd-bb57fc7d7-qq4p8"]
Sep 09 21:13:30.797897 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:30.797855    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="310677ad-9164-44c5-bee9-0fcde74291bc" path="/var/lib/kubelet/pods/310677ad-9164-44c5-bee9-0fcde74291bc/volumes"
Sep 09 21:13:32.714093 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:32.714054    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb" podUID="0e114a8f-50ad-4b31-a44c-0a146366ee71" containerName="ensemble-graph-71ac5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:13:37.714815 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:37.714770    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb" podUID="0e114a8f-50ad-4b31-a44c-0a146366ee71" containerName="ensemble-graph-71ac5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:13:42.714344 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:42.714296    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb" podUID="0e114a8f-50ad-4b31-a44c-0a146366ee71" containerName="ensemble-graph-71ac5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:13:42.714725 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:42.714403    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"
Sep 09 21:13:47.714155 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:47.714112    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb" podUID="0e114a8f-50ad-4b31-a44c-0a146366ee71" containerName="ensemble-graph-71ac5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:13:52.714898 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:52.714849    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb" podUID="0e114a8f-50ad-4b31-a44c-0a146366ee71" containerName="ensemble-graph-71ac5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:13:57.714848 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:57.714810    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb" podUID="0e114a8f-50ad-4b31-a44c-0a146366ee71" containerName="ensemble-graph-71ac5" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:13:59.125793 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.125760    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"]
Sep 09 21:13:59.126160 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.126090    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="310677ad-9164-44c5-bee9-0fcde74291bc" containerName="sequence-graph-456dd"
Sep 09 21:13:59.126160 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.126102    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="310677ad-9164-44c5-bee9-0fcde74291bc" containerName="sequence-graph-456dd"
Sep 09 21:13:59.126160 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.126150    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="310677ad-9164-44c5-bee9-0fcde74291bc" containerName="sequence-graph-456dd"
Sep 09 21:13:59.128833 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.128813    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"
Sep 09 21:13:59.132002 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.131982    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-07fe8-serving-cert\""
Sep 09 21:13:59.132113 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.131983    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-07fe8-kube-rbac-proxy-sar-config\""
Sep 09 21:13:59.146311 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.146279    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"]
Sep 09 21:13:59.225487 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.225452    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc057e60-fc93-4bf1-9abf-bc474ebdadd0-proxy-tls\") pod \"sequence-graph-07fe8-69b4c4c9d8-q5tmf\" (UID: \"dc057e60-fc93-4bf1-9abf-bc474ebdadd0\") " pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"
Sep 09 21:13:59.225662 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.225513    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc057e60-fc93-4bf1-9abf-bc474ebdadd0-openshift-service-ca-bundle\") pod \"sequence-graph-07fe8-69b4c4c9d8-q5tmf\" (UID: \"dc057e60-fc93-4bf1-9abf-bc474ebdadd0\") " pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"
Sep 09 21:13:59.326423 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.326381    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc057e60-fc93-4bf1-9abf-bc474ebdadd0-proxy-tls\") pod \"sequence-graph-07fe8-69b4c4c9d8-q5tmf\" (UID: \"dc057e60-fc93-4bf1-9abf-bc474ebdadd0\") " pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"
Sep 09 21:13:59.326587 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.326446    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc057e60-fc93-4bf1-9abf-bc474ebdadd0-openshift-service-ca-bundle\") pod \"sequence-graph-07fe8-69b4c4c9d8-q5tmf\" (UID: \"dc057e60-fc93-4bf1-9abf-bc474ebdadd0\") " pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"
Sep 09 21:13:59.326587 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:13:59.326539    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-07fe8-serving-cert: secret "sequence-graph-07fe8-serving-cert" not found
Sep 09 21:13:59.326659 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:13:59.326636    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc057e60-fc93-4bf1-9abf-bc474ebdadd0-proxy-tls podName:dc057e60-fc93-4bf1-9abf-bc474ebdadd0 nodeName:}" failed. No retries permitted until 2026-09-09 21:13:59.826618604 +0000 UTC m=+1291.497248286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/dc057e60-fc93-4bf1-9abf-bc474ebdadd0-proxy-tls") pod "sequence-graph-07fe8-69b4c4c9d8-q5tmf" (UID: "dc057e60-fc93-4bf1-9abf-bc474ebdadd0") : secret "sequence-graph-07fe8-serving-cert" not found
Sep 09 21:13:59.327071 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.327054    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc057e60-fc93-4bf1-9abf-bc474ebdadd0-openshift-service-ca-bundle\") pod \"sequence-graph-07fe8-69b4c4c9d8-q5tmf\" (UID: \"dc057e60-fc93-4bf1-9abf-bc474ebdadd0\") " pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"
Sep 09 21:13:59.824378 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.824350    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"
Sep 09 21:13:59.831168 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.831138    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc057e60-fc93-4bf1-9abf-bc474ebdadd0-proxy-tls\") pod \"sequence-graph-07fe8-69b4c4c9d8-q5tmf\" (UID: \"dc057e60-fc93-4bf1-9abf-bc474ebdadd0\") " pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"
Sep 09 21:13:59.833418 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.833391    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc057e60-fc93-4bf1-9abf-bc474ebdadd0-proxy-tls\") pod \"sequence-graph-07fe8-69b4c4c9d8-q5tmf\" (UID: \"dc057e60-fc93-4bf1-9abf-bc474ebdadd0\") " pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"
Sep 09 21:13:59.851206 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.851169    2585 generic.go:356] "Generic (PLEG): container finished" podID="0e114a8f-50ad-4b31-a44c-0a146366ee71" containerID="bd5db2bb87f024777482bcfcf8afeb1ab5e8f22b70d2f4bb0b1091eae64ee590" exitCode=0
Sep 09 21:13:59.851359 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.851215    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"
Sep 09 21:13:59.851359 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.851236    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb" event={"ID":"0e114a8f-50ad-4b31-a44c-0a146366ee71","Type":"ContainerDied","Data":"bd5db2bb87f024777482bcfcf8afeb1ab5e8f22b70d2f4bb0b1091eae64ee590"}
Sep 09 21:13:59.851359 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.851307    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb" event={"ID":"0e114a8f-50ad-4b31-a44c-0a146366ee71","Type":"ContainerDied","Data":"58768ad519a1593ae54b895f5f619a42e6e85622be6b4d873782376e1272f7de"}
Sep 09 21:13:59.851359 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.851332    2585 scope.go:117] "RemoveContainer" containerID="bd5db2bb87f024777482bcfcf8afeb1ab5e8f22b70d2f4bb0b1091eae64ee590"
Sep 09 21:13:59.859286 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.859269    2585 scope.go:117] "RemoveContainer" containerID="bd5db2bb87f024777482bcfcf8afeb1ab5e8f22b70d2f4bb0b1091eae64ee590"
Sep 09 21:13:59.859556 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:13:59.859539    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd5db2bb87f024777482bcfcf8afeb1ab5e8f22b70d2f4bb0b1091eae64ee590\": container with ID starting with bd5db2bb87f024777482bcfcf8afeb1ab5e8f22b70d2f4bb0b1091eae64ee590 not found: ID does not exist" containerID="bd5db2bb87f024777482bcfcf8afeb1ab5e8f22b70d2f4bb0b1091eae64ee590"
Sep 09 21:13:59.859602 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.859563    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd5db2bb87f024777482bcfcf8afeb1ab5e8f22b70d2f4bb0b1091eae64ee590"} err="failed to get container status \"bd5db2bb87f024777482bcfcf8afeb1ab5e8f22b70d2f4bb0b1091eae64ee590\": rpc error: code = NotFound desc = could not find container \"bd5db2bb87f024777482bcfcf8afeb1ab5e8f22b70d2f4bb0b1091eae64ee590\": container with ID starting with bd5db2bb87f024777482bcfcf8afeb1ab5e8f22b70d2f4bb0b1091eae64ee590 not found: ID does not exist"
Sep 09 21:13:59.931677 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.931642    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0e114a8f-50ad-4b31-a44c-0a146366ee71-proxy-tls\") pod \"0e114a8f-50ad-4b31-a44c-0a146366ee71\" (UID: \"0e114a8f-50ad-4b31-a44c-0a146366ee71\") "
Sep 09 21:13:59.931844 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.931696    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e114a8f-50ad-4b31-a44c-0a146366ee71-openshift-service-ca-bundle\") pod \"0e114a8f-50ad-4b31-a44c-0a146366ee71\" (UID: \"0e114a8f-50ad-4b31-a44c-0a146366ee71\") "
Sep 09 21:13:59.932141 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.932111    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e114a8f-50ad-4b31-a44c-0a146366ee71-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "0e114a8f-50ad-4b31-a44c-0a146366ee71" (UID: "0e114a8f-50ad-4b31-a44c-0a146366ee71"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:13:59.933814 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:13:59.933769    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e114a8f-50ad-4b31-a44c-0a146366ee71-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0e114a8f-50ad-4b31-a44c-0a146366ee71" (UID: "0e114a8f-50ad-4b31-a44c-0a146366ee71"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:14:00.033011 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:00.032970    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0e114a8f-50ad-4b31-a44c-0a146366ee71-proxy-tls\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:14:00.033011 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:00.033005    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e114a8f-50ad-4b31-a44c-0a146366ee71-openshift-service-ca-bundle\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:14:00.038937 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:00.038914    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"
Sep 09 21:14:00.175305 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:00.175277    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"]
Sep 09 21:14:00.178941 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:00.178912    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-71ac5-5b87cdfb8f-d9rgb"]
Sep 09 21:14:00.313422 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:00.313398    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"]
Sep 09 21:14:00.315964 ip-10-0-129-24 kubenswrapper[2585]: W0909 21:14:00.315938    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc057e60_fc93_4bf1_9abf_bc474ebdadd0.slice/crio-398b3c4029363695654eba86fed42909beccc47d88b80dd26b7ea83836faec26 WatchSource:0}: Error finding container 398b3c4029363695654eba86fed42909beccc47d88b80dd26b7ea83836faec26: Status 404 returned error can't find the container with id 398b3c4029363695654eba86fed42909beccc47d88b80dd26b7ea83836faec26
Sep 09 21:14:00.798528 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:00.798497    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e114a8f-50ad-4b31-a44c-0a146366ee71" path="/var/lib/kubelet/pods/0e114a8f-50ad-4b31-a44c-0a146366ee71/volumes"
Sep 09 21:14:00.856737 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:00.856701    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf" event={"ID":"dc057e60-fc93-4bf1-9abf-bc474ebdadd0","Type":"ContainerStarted","Data":"016f74c53320c004a5262f0c206f273b0db553bbd0bc5154cbc5f37fad9649a3"}
Sep 09 21:14:00.856939 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:00.856762    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf" event={"ID":"dc057e60-fc93-4bf1-9abf-bc474ebdadd0","Type":"ContainerStarted","Data":"398b3c4029363695654eba86fed42909beccc47d88b80dd26b7ea83836faec26"}
Sep 09 21:14:00.856939 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:00.856791    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"
Sep 09 21:14:00.896023 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:00.895971    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf" podStartSLOduration=1.8959542329999999 podStartE2EDuration="1.895954233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:13:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:00.895631431 +0000 UTC m=+1292.566261131" watchObservedRunningTime="2026-09-09 21:14:00.895954233 +0000 UTC m=+1292.566583933"
Sep 09 21:14:06.867709 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:06.867679    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"
Sep 09 21:14:09.229915 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:09.229870    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"]
Sep 09 21:14:09.230322 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:09.230072    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf" podUID="dc057e60-fc93-4bf1-9abf-bc474ebdadd0" containerName="sequence-graph-07fe8" containerID="cri-o://016f74c53320c004a5262f0c206f273b0db553bbd0bc5154cbc5f37fad9649a3" gracePeriod=30
Sep 09 21:14:11.866096 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:11.866052    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf" podUID="dc057e60-fc93-4bf1-9abf-bc474ebdadd0" containerName="sequence-graph-07fe8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:14:16.866767 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:16.866724    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf" podUID="dc057e60-fc93-4bf1-9abf-bc474ebdadd0" containerName="sequence-graph-07fe8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:14:21.866278 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:21.866234    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf" podUID="dc057e60-fc93-4bf1-9abf-bc474ebdadd0" containerName="sequence-graph-07fe8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:14:21.866652 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:21.866350    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"
Sep 09 21:14:26.866004 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:26.865958    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf" podUID="dc057e60-fc93-4bf1-9abf-bc474ebdadd0" containerName="sequence-graph-07fe8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:14:31.866153 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:31.866105    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf" podUID="dc057e60-fc93-4bf1-9abf-bc474ebdadd0" containerName="sequence-graph-07fe8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:14:36.866053 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:36.866013    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf" podUID="dc057e60-fc93-4bf1-9abf-bc474ebdadd0" containerName="sequence-graph-07fe8" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:14:39.370968 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:39.370944    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"
Sep 09 21:14:39.437826 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:39.437798    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc057e60-fc93-4bf1-9abf-bc474ebdadd0-proxy-tls\") pod \"dc057e60-fc93-4bf1-9abf-bc474ebdadd0\" (UID: \"dc057e60-fc93-4bf1-9abf-bc474ebdadd0\") "
Sep 09 21:14:39.437974 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:39.437904    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc057e60-fc93-4bf1-9abf-bc474ebdadd0-openshift-service-ca-bundle\") pod \"dc057e60-fc93-4bf1-9abf-bc474ebdadd0\" (UID: \"dc057e60-fc93-4bf1-9abf-bc474ebdadd0\") "
Sep 09 21:14:39.438297 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:39.438272    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc057e60-fc93-4bf1-9abf-bc474ebdadd0-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "dc057e60-fc93-4bf1-9abf-bc474ebdadd0" (UID: "dc057e60-fc93-4bf1-9abf-bc474ebdadd0"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:14:39.439865 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:39.439838    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc057e60-fc93-4bf1-9abf-bc474ebdadd0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "dc057e60-fc93-4bf1-9abf-bc474ebdadd0" (UID: "dc057e60-fc93-4bf1-9abf-bc474ebdadd0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:14:39.538982 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:39.538958    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc057e60-fc93-4bf1-9abf-bc474ebdadd0-openshift-service-ca-bundle\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:14:39.538982 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:39.538982    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc057e60-fc93-4bf1-9abf-bc474ebdadd0-proxy-tls\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:14:40.005216 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.005128    2585 generic.go:356] "Generic (PLEG): container finished" podID="dc057e60-fc93-4bf1-9abf-bc474ebdadd0" containerID="016f74c53320c004a5262f0c206f273b0db553bbd0bc5154cbc5f37fad9649a3" exitCode=0
Sep 09 21:14:40.005216 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.005173    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"
Sep 09 21:14:40.005423 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.005229    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf" event={"ID":"dc057e60-fc93-4bf1-9abf-bc474ebdadd0","Type":"ContainerDied","Data":"016f74c53320c004a5262f0c206f273b0db553bbd0bc5154cbc5f37fad9649a3"}
Sep 09 21:14:40.005423 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.005279    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf" event={"ID":"dc057e60-fc93-4bf1-9abf-bc474ebdadd0","Type":"ContainerDied","Data":"398b3c4029363695654eba86fed42909beccc47d88b80dd26b7ea83836faec26"}
Sep 09 21:14:40.005423 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.005302    2585 scope.go:117] "RemoveContainer" containerID="016f74c53320c004a5262f0c206f273b0db553bbd0bc5154cbc5f37fad9649a3"
Sep 09 21:14:40.013534 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.013516    2585 scope.go:117] "RemoveContainer" containerID="016f74c53320c004a5262f0c206f273b0db553bbd0bc5154cbc5f37fad9649a3"
Sep 09 21:14:40.013771 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:14:40.013752    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"016f74c53320c004a5262f0c206f273b0db553bbd0bc5154cbc5f37fad9649a3\": container with ID starting with 016f74c53320c004a5262f0c206f273b0db553bbd0bc5154cbc5f37fad9649a3 not found: ID does not exist" containerID="016f74c53320c004a5262f0c206f273b0db553bbd0bc5154cbc5f37fad9649a3"
Sep 09 21:14:40.013822 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.013777    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"016f74c53320c004a5262f0c206f273b0db553bbd0bc5154cbc5f37fad9649a3"} err="failed to get container status \"016f74c53320c004a5262f0c206f273b0db553bbd0bc5154cbc5f37fad9649a3\": rpc error: code = NotFound desc = could not find container \"016f74c53320c004a5262f0c206f273b0db553bbd0bc5154cbc5f37fad9649a3\": container with ID starting with 016f74c53320c004a5262f0c206f273b0db553bbd0bc5154cbc5f37fad9649a3 not found: ID does not exist"
Sep 09 21:14:40.029386 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.029361    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"]
Sep 09 21:14:40.031055 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.031034    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-07fe8-69b4c4c9d8-q5tmf"]
Sep 09 21:14:40.139240 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.139205    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"]
Sep 09 21:14:40.139518 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.139504    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e114a8f-50ad-4b31-a44c-0a146366ee71" containerName="ensemble-graph-71ac5"
Sep 09 21:14:40.139564 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.139520    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e114a8f-50ad-4b31-a44c-0a146366ee71" containerName="ensemble-graph-71ac5"
Sep 09 21:14:40.139564 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.139533    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc057e60-fc93-4bf1-9abf-bc474ebdadd0" containerName="sequence-graph-07fe8"
Sep 09 21:14:40.139564 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.139539    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc057e60-fc93-4bf1-9abf-bc474ebdadd0" containerName="sequence-graph-07fe8"
Sep 09 21:14:40.139655 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.139589    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e114a8f-50ad-4b31-a44c-0a146366ee71" containerName="ensemble-graph-71ac5"
Sep 09 21:14:40.139655 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.139597    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc057e60-fc93-4bf1-9abf-bc474ebdadd0" containerName="sequence-graph-07fe8"
Sep 09 21:14:40.143582 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.143560    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"
Sep 09 21:14:40.146308 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.146290    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-2e5cb-kube-rbac-proxy-sar-config\""
Sep 09 21:14:40.147143 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.147123    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-2e5cb-serving-cert\""
Sep 09 21:14:40.147442 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.147419    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 09 21:14:40.147564 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.147492    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-c7qlq\""
Sep 09 21:14:40.158699 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.158677    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"]
Sep 09 21:14:40.244743 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.244709    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bcc183ba-b47b-403a-b4be-39f2ced850e2-proxy-tls\") pod \"ensemble-graph-2e5cb-67c7f87c5c-wp89n\" (UID: \"bcc183ba-b47b-403a-b4be-39f2ced850e2\") " pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"
Sep 09 21:14:40.244933 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.244767    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcc183ba-b47b-403a-b4be-39f2ced850e2-openshift-service-ca-bundle\") pod \"ensemble-graph-2e5cb-67c7f87c5c-wp89n\" (UID: \"bcc183ba-b47b-403a-b4be-39f2ced850e2\") " pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"
Sep 09 21:14:40.346010 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.345967    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcc183ba-b47b-403a-b4be-39f2ced850e2-openshift-service-ca-bundle\") pod \"ensemble-graph-2e5cb-67c7f87c5c-wp89n\" (UID: \"bcc183ba-b47b-403a-b4be-39f2ced850e2\") " pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"
Sep 09 21:14:40.346219 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.346035    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bcc183ba-b47b-403a-b4be-39f2ced850e2-proxy-tls\") pod \"ensemble-graph-2e5cb-67c7f87c5c-wp89n\" (UID: \"bcc183ba-b47b-403a-b4be-39f2ced850e2\") " pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"
Sep 09 21:14:40.346219 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:14:40.346137    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-2e5cb-serving-cert: secret "ensemble-graph-2e5cb-serving-cert" not found
Sep 09 21:14:40.346219 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:14:40.346191    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcc183ba-b47b-403a-b4be-39f2ced850e2-proxy-tls podName:bcc183ba-b47b-403a-b4be-39f2ced850e2 nodeName:}" failed. No retries permitted until 2026-09-09 21:14:40.84617574 +0000 UTC m=+1332.516805418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/bcc183ba-b47b-403a-b4be-39f2ced850e2-proxy-tls") pod "ensemble-graph-2e5cb-67c7f87c5c-wp89n" (UID: "bcc183ba-b47b-403a-b4be-39f2ced850e2") : secret "ensemble-graph-2e5cb-serving-cert" not found
Sep 09 21:14:40.346702 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.346677    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcc183ba-b47b-403a-b4be-39f2ced850e2-openshift-service-ca-bundle\") pod \"ensemble-graph-2e5cb-67c7f87c5c-wp89n\" (UID: \"bcc183ba-b47b-403a-b4be-39f2ced850e2\") " pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"
Sep 09 21:14:40.798623 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.798588    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc057e60-fc93-4bf1-9abf-bc474ebdadd0" path="/var/lib/kubelet/pods/dc057e60-fc93-4bf1-9abf-bc474ebdadd0/volumes"
Sep 09 21:14:40.849833 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.849797    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bcc183ba-b47b-403a-b4be-39f2ced850e2-proxy-tls\") pod \"ensemble-graph-2e5cb-67c7f87c5c-wp89n\" (UID: \"bcc183ba-b47b-403a-b4be-39f2ced850e2\") " pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"
Sep 09 21:14:40.852251 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:40.852231    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bcc183ba-b47b-403a-b4be-39f2ced850e2-proxy-tls\") pod \"ensemble-graph-2e5cb-67c7f87c5c-wp89n\" (UID: \"bcc183ba-b47b-403a-b4be-39f2ced850e2\") " pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"
Sep 09 21:14:41.053001 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:41.052915    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"
Sep 09 21:14:41.330694 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:41.330666    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"]
Sep 09 21:14:41.332699 ip-10-0-129-24 kubenswrapper[2585]: W0909 21:14:41.332674    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcc183ba_b47b_403a_b4be_39f2ced850e2.slice/crio-33ffc2c3936fe12bca7f68fbf78208eca326bf60c986248af910f76ebfc57885 WatchSource:0}: Error finding container 33ffc2c3936fe12bca7f68fbf78208eca326bf60c986248af910f76ebfc57885: Status 404 returned error can't find the container with id 33ffc2c3936fe12bca7f68fbf78208eca326bf60c986248af910f76ebfc57885
Sep 09 21:14:42.014453 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:42.014412    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n" event={"ID":"bcc183ba-b47b-403a-b4be-39f2ced850e2","Type":"ContainerStarted","Data":"110f339771fc703ab6fd65d16be1d0f01c691aae3bfd45b6b148fe74b1d71eac"}
Sep 09 21:14:42.014840 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:42.014467    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n" event={"ID":"bcc183ba-b47b-403a-b4be-39f2ced850e2","Type":"ContainerStarted","Data":"33ffc2c3936fe12bca7f68fbf78208eca326bf60c986248af910f76ebfc57885"}
Sep 09 21:14:42.014840 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:42.014506    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"
Sep 09 21:14:42.040226 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:42.040176    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n" podStartSLOduration=2.040158249 podStartE2EDuration="2.040158249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:42.039634423 +0000 UTC m=+1333.710264129" watchObservedRunningTime="2026-09-09 21:14:42.040158249 +0000 UTC m=+1333.710787988"
Sep 09 21:14:48.023768 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:14:48.023734    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"
Sep 09 21:15:09.535236 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:09.535152    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"]
Sep 09 21:15:09.538600 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:09.538582    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"
Sep 09 21:15:09.541869 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:09.541840    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-0dc3b-serving-cert\""
Sep 09 21:15:09.541869 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:09.541853    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-0dc3b-kube-rbac-proxy-sar-config\""
Sep 09 21:15:09.573628 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:09.573596    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"]
Sep 09 21:15:09.678325 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:09.678287    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c-openshift-service-ca-bundle\") pod \"sequence-graph-0dc3b-59c974478f-9s5ds\" (UID: \"30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c\") " pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"
Sep 09 21:15:09.678325 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:09.678331    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c-proxy-tls\") pod \"sequence-graph-0dc3b-59c974478f-9s5ds\" (UID: \"30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c\") " pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"
Sep 09 21:15:09.779367 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:09.779326    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c-openshift-service-ca-bundle\") pod \"sequence-graph-0dc3b-59c974478f-9s5ds\" (UID: \"30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c\") " pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"
Sep 09 21:15:09.779367 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:09.779378    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c-proxy-tls\") pod \"sequence-graph-0dc3b-59c974478f-9s5ds\" (UID: \"30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c\") " pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"
Sep 09 21:15:09.779580 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:15:09.779491    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-0dc3b-serving-cert: secret "sequence-graph-0dc3b-serving-cert" not found
Sep 09 21:15:09.779580 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:15:09.779549    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c-proxy-tls podName:30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c nodeName:}" failed. No retries permitted until 2026-09-09 21:15:10.279532031 +0000 UTC m=+1361.950161708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c-proxy-tls") pod "sequence-graph-0dc3b-59c974478f-9s5ds" (UID: "30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c") : secret "sequence-graph-0dc3b-serving-cert" not found
Sep 09 21:15:09.780020 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:09.779998    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c-openshift-service-ca-bundle\") pod \"sequence-graph-0dc3b-59c974478f-9s5ds\" (UID: \"30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c\") " pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"
Sep 09 21:15:10.284030 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:10.283994    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c-proxy-tls\") pod \"sequence-graph-0dc3b-59c974478f-9s5ds\" (UID: \"30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c\") " pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"
Sep 09 21:15:10.286517 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:10.286491    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c-proxy-tls\") pod \"sequence-graph-0dc3b-59c974478f-9s5ds\" (UID: \"30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c\") " pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"
Sep 09 21:15:10.448340 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:10.448300    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"
Sep 09 21:15:10.724854 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:10.724769    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"]
Sep 09 21:15:10.727785 ip-10-0-129-24 kubenswrapper[2585]: W0909 21:15:10.727752    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30c2e7f3_ae8a_4dae_8f2c_4bbcee274d4c.slice/crio-d7d2c585619a366d625e74db7000bc19e7e01da80d38a3d4e3427d0b5efffabb WatchSource:0}: Error finding container d7d2c585619a366d625e74db7000bc19e7e01da80d38a3d4e3427d0b5efffabb: Status 404 returned error can't find the container with id d7d2c585619a366d625e74db7000bc19e7e01da80d38a3d4e3427d0b5efffabb
Sep 09 21:15:11.122636 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:11.122601    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds" event={"ID":"30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c","Type":"ContainerStarted","Data":"fbfba22b5e60a95b8788e373a7d792b83127f462c0477d2c44845a13c46c7705"}
Sep 09 21:15:11.122808 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:11.122653    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds" event={"ID":"30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c","Type":"ContainerStarted","Data":"d7d2c585619a366d625e74db7000bc19e7e01da80d38a3d4e3427d0b5efffabb"}
Sep 09 21:15:11.122808 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:11.122693    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"
Sep 09 21:15:11.147232 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:11.147177    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds" podStartSLOduration=2.147161736 podStartE2EDuration="2.147161736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:15:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:15:11.144405357 +0000 UTC m=+1362.815035058" watchObservedRunningTime="2026-09-09 21:15:11.147161736 +0000 UTC m=+1362.817791436"
Sep 09 21:15:17.131150 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:15:17.131122    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"
Sep 09 21:22:54.728698 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:22:54.728664    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"]
Sep 09 21:22:54.729206 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:22:54.728866    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n" podUID="bcc183ba-b47b-403a-b4be-39f2ced850e2" containerName="ensemble-graph-2e5cb" containerID="cri-o://110f339771fc703ab6fd65d16be1d0f01c691aae3bfd45b6b148fe74b1d71eac" gracePeriod=30
Sep 09 21:22:58.022167 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:22:58.022122    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n" podUID="bcc183ba-b47b-403a-b4be-39f2ced850e2" containerName="ensemble-graph-2e5cb" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:23:03.022458 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:03.022414    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n" podUID="bcc183ba-b47b-403a-b4be-39f2ced850e2" containerName="ensemble-graph-2e5cb" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:23:08.022662 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:08.022614    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n" podUID="bcc183ba-b47b-403a-b4be-39f2ced850e2" containerName="ensemble-graph-2e5cb" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:23:08.023070 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:08.022727    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"
Sep 09 21:23:13.022664 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:13.022620    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n" podUID="bcc183ba-b47b-403a-b4be-39f2ced850e2" containerName="ensemble-graph-2e5cb" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:23:18.022774 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:18.022722    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n" podUID="bcc183ba-b47b-403a-b4be-39f2ced850e2" containerName="ensemble-graph-2e5cb" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:23:23.021819 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:23.021777    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n" podUID="bcc183ba-b47b-403a-b4be-39f2ced850e2" containerName="ensemble-graph-2e5cb" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:23:24.133004 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:24.132972    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"]
Sep 09 21:23:24.133392 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:24.133289    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds" podUID="30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c" containerName="sequence-graph-0dc3b" containerID="cri-o://fbfba22b5e60a95b8788e373a7d792b83127f462c0477d2c44845a13c46c7705" gracePeriod=30
Sep 09 21:23:24.883517 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:24.883493    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"
Sep 09 21:23:24.939323 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:24.939293    2585 generic.go:356] "Generic (PLEG): container finished" podID="bcc183ba-b47b-403a-b4be-39f2ced850e2" containerID="110f339771fc703ab6fd65d16be1d0f01c691aae3bfd45b6b148fe74b1d71eac" exitCode=0
Sep 09 21:23:24.939492 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:24.939338    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"
Sep 09 21:23:24.939492 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:24.939385    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n" event={"ID":"bcc183ba-b47b-403a-b4be-39f2ced850e2","Type":"ContainerDied","Data":"110f339771fc703ab6fd65d16be1d0f01c691aae3bfd45b6b148fe74b1d71eac"}
Sep 09 21:23:24.939492 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:24.939435    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n" event={"ID":"bcc183ba-b47b-403a-b4be-39f2ced850e2","Type":"ContainerDied","Data":"33ffc2c3936fe12bca7f68fbf78208eca326bf60c986248af910f76ebfc57885"}
Sep 09 21:23:24.939492 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:24.939459    2585 scope.go:117] "RemoveContainer" containerID="110f339771fc703ab6fd65d16be1d0f01c691aae3bfd45b6b148fe74b1d71eac"
Sep 09 21:23:24.947314 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:24.947297    2585 scope.go:117] "RemoveContainer" containerID="110f339771fc703ab6fd65d16be1d0f01c691aae3bfd45b6b148fe74b1d71eac"
Sep 09 21:23:24.947616 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:23:24.947599    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"110f339771fc703ab6fd65d16be1d0f01c691aae3bfd45b6b148fe74b1d71eac\": container with ID starting with 110f339771fc703ab6fd65d16be1d0f01c691aae3bfd45b6b148fe74b1d71eac not found: ID does not exist" containerID="110f339771fc703ab6fd65d16be1d0f01c691aae3bfd45b6b148fe74b1d71eac"
Sep 09 21:23:24.947657 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:24.947624    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"110f339771fc703ab6fd65d16be1d0f01c691aae3bfd45b6b148fe74b1d71eac"} err="failed to get container status \"110f339771fc703ab6fd65d16be1d0f01c691aae3bfd45b6b148fe74b1d71eac\": rpc error: code = NotFound desc = could not find container \"110f339771fc703ab6fd65d16be1d0f01c691aae3bfd45b6b148fe74b1d71eac\": container with ID starting with 110f339771fc703ab6fd65d16be1d0f01c691aae3bfd45b6b148fe74b1d71eac not found: ID does not exist"
Sep 09 21:23:24.963894 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:24.963859    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcc183ba-b47b-403a-b4be-39f2ced850e2-openshift-service-ca-bundle\") pod \"bcc183ba-b47b-403a-b4be-39f2ced850e2\" (UID: \"bcc183ba-b47b-403a-b4be-39f2ced850e2\") "
Sep 09 21:23:24.963988 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:24.963943    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bcc183ba-b47b-403a-b4be-39f2ced850e2-proxy-tls\") pod \"bcc183ba-b47b-403a-b4be-39f2ced850e2\" (UID: \"bcc183ba-b47b-403a-b4be-39f2ced850e2\") "
Sep 09 21:23:24.964164 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:24.964145    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcc183ba-b47b-403a-b4be-39f2ced850e2-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "bcc183ba-b47b-403a-b4be-39f2ced850e2" (UID: "bcc183ba-b47b-403a-b4be-39f2ced850e2"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:23:24.965843 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:24.965824    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcc183ba-b47b-403a-b4be-39f2ced850e2-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bcc183ba-b47b-403a-b4be-39f2ced850e2" (UID: "bcc183ba-b47b-403a-b4be-39f2ced850e2"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:23:25.064866 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:25.064832    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bcc183ba-b47b-403a-b4be-39f2ced850e2-proxy-tls\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:23:25.064866 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:25.064865    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bcc183ba-b47b-403a-b4be-39f2ced850e2-openshift-service-ca-bundle\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:23:25.261351 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:25.261319    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"]
Sep 09 21:23:25.267558 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:25.267528    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2e5cb-67c7f87c5c-wp89n"]
Sep 09 21:23:26.797870 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:26.797837    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcc183ba-b47b-403a-b4be-39f2ced850e2" path="/var/lib/kubelet/pods/bcc183ba-b47b-403a-b4be-39f2ced850e2/volumes"
Sep 09 21:23:27.130024 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:27.129938    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds" podUID="30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c" containerName="sequence-graph-0dc3b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:23:32.130099 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:32.130055    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds" podUID="30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c" containerName="sequence-graph-0dc3b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:23:37.129908 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:37.129849    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds" podUID="30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c" containerName="sequence-graph-0dc3b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:23:37.130384 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:37.129987    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"
Sep 09 21:23:42.129992 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:42.129949    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds" podUID="30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c" containerName="sequence-graph-0dc3b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:23:47.130134 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:47.130096    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds" podUID="30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c" containerName="sequence-graph-0dc3b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:23:52.129964 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:52.129861    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds" podUID="30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c" containerName="sequence-graph-0dc3b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:23:54.766931 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:54.766906    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"
Sep 09 21:23:54.787842 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:54.787808    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c-proxy-tls\") pod \"30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c\" (UID: \"30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c\") "
Sep 09 21:23:54.788013 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:54.787915    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c-openshift-service-ca-bundle\") pod \"30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c\" (UID: \"30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c\") "
Sep 09 21:23:54.788321 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:54.788294    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c" (UID: "30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:23:54.789908 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:54.789867    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c" (UID: "30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:23:54.888462 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:54.888430    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c-openshift-service-ca-bundle\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:23:54.888462 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:54.888459    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c-proxy-tls\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:23:55.014013 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.013939    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"]
Sep 09 21:23:55.014248 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.014236    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcc183ba-b47b-403a-b4be-39f2ced850e2" containerName="ensemble-graph-2e5cb"
Sep 09 21:23:55.014290 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.014249    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcc183ba-b47b-403a-b4be-39f2ced850e2" containerName="ensemble-graph-2e5cb"
Sep 09 21:23:55.014290 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.014270    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c" containerName="sequence-graph-0dc3b"
Sep 09 21:23:55.014290 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.014276    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c" containerName="sequence-graph-0dc3b"
Sep 09 21:23:55.014415 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.014321    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcc183ba-b47b-403a-b4be-39f2ced850e2" containerName="ensemble-graph-2e5cb"
Sep 09 21:23:55.014415 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.014330    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c" containerName="sequence-graph-0dc3b"
Sep 09 21:23:55.017273 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.017251    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"
Sep 09 21:23:55.020130 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.020101    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-78161-kube-rbac-proxy-sar-config\""
Sep 09 21:23:55.020242 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.020135    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-78161-serving-cert\""
Sep 09 21:23:55.030031 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.030011    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"]
Sep 09 21:23:55.052215 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.052191    2585 generic.go:356] "Generic (PLEG): container finished" podID="30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c" containerID="fbfba22b5e60a95b8788e373a7d792b83127f462c0477d2c44845a13c46c7705" exitCode=0
Sep 09 21:23:55.052312 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.052257    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"
Sep 09 21:23:55.052312 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.052277    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds" event={"ID":"30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c","Type":"ContainerDied","Data":"fbfba22b5e60a95b8788e373a7d792b83127f462c0477d2c44845a13c46c7705"}
Sep 09 21:23:55.052387 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.052321    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds" event={"ID":"30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c","Type":"ContainerDied","Data":"d7d2c585619a366d625e74db7000bc19e7e01da80d38a3d4e3427d0b5efffabb"}
Sep 09 21:23:55.052387 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.052349    2585 scope.go:117] "RemoveContainer" containerID="fbfba22b5e60a95b8788e373a7d792b83127f462c0477d2c44845a13c46c7705"
Sep 09 21:23:55.060015 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.059995    2585 scope.go:117] "RemoveContainer" containerID="fbfba22b5e60a95b8788e373a7d792b83127f462c0477d2c44845a13c46c7705"
Sep 09 21:23:55.060297 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:23:55.060273    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbfba22b5e60a95b8788e373a7d792b83127f462c0477d2c44845a13c46c7705\": container with ID starting with fbfba22b5e60a95b8788e373a7d792b83127f462c0477d2c44845a13c46c7705 not found: ID does not exist" containerID="fbfba22b5e60a95b8788e373a7d792b83127f462c0477d2c44845a13c46c7705"
Sep 09 21:23:55.060366 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.060309    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbfba22b5e60a95b8788e373a7d792b83127f462c0477d2c44845a13c46c7705"} err="failed to get container status \"fbfba22b5e60a95b8788e373a7d792b83127f462c0477d2c44845a13c46c7705\": rpc error: code = NotFound desc = could not find container \"fbfba22b5e60a95b8788e373a7d792b83127f462c0477d2c44845a13c46c7705\": container with ID starting with fbfba22b5e60a95b8788e373a7d792b83127f462c0477d2c44845a13c46c7705 not found: ID does not exist"
Sep 09 21:23:55.071335 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.071312    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"]
Sep 09 21:23:55.078867 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.078845    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-0dc3b-59c974478f-9s5ds"]
Sep 09 21:23:55.090009 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.089985    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe6dbce6-a06a-4c6c-a910-0dd83135bfda-proxy-tls\") pod \"splitter-graph-78161-df5b9dc88-hxgtn\" (UID: \"fe6dbce6-a06a-4c6c-a910-0dd83135bfda\") " pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"
Sep 09 21:23:55.090109 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.090022    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe6dbce6-a06a-4c6c-a910-0dd83135bfda-openshift-service-ca-bundle\") pod \"splitter-graph-78161-df5b9dc88-hxgtn\" (UID: \"fe6dbce6-a06a-4c6c-a910-0dd83135bfda\") " pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"
Sep 09 21:23:55.191181 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.191145    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe6dbce6-a06a-4c6c-a910-0dd83135bfda-proxy-tls\") pod \"splitter-graph-78161-df5b9dc88-hxgtn\" (UID: \"fe6dbce6-a06a-4c6c-a910-0dd83135bfda\") " pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"
Sep 09 21:23:55.191360 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.191189    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe6dbce6-a06a-4c6c-a910-0dd83135bfda-openshift-service-ca-bundle\") pod \"splitter-graph-78161-df5b9dc88-hxgtn\" (UID: \"fe6dbce6-a06a-4c6c-a910-0dd83135bfda\") " pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"
Sep 09 21:23:55.191360 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:23:55.191317    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-78161-serving-cert: secret "splitter-graph-78161-serving-cert" not found
Sep 09 21:23:55.191487 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:23:55.191448    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe6dbce6-a06a-4c6c-a910-0dd83135bfda-proxy-tls podName:fe6dbce6-a06a-4c6c-a910-0dd83135bfda nodeName:}" failed. No retries permitted until 2026-09-09 21:23:55.691422094 +0000 UTC m=+1887.362051777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/fe6dbce6-a06a-4c6c-a910-0dd83135bfda-proxy-tls") pod "splitter-graph-78161-df5b9dc88-hxgtn" (UID: "fe6dbce6-a06a-4c6c-a910-0dd83135bfda") : secret "splitter-graph-78161-serving-cert" not found
Sep 09 21:23:55.191797 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.191778    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe6dbce6-a06a-4c6c-a910-0dd83135bfda-openshift-service-ca-bundle\") pod \"splitter-graph-78161-df5b9dc88-hxgtn\" (UID: \"fe6dbce6-a06a-4c6c-a910-0dd83135bfda\") " pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"
Sep 09 21:23:55.694374 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.694335    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe6dbce6-a06a-4c6c-a910-0dd83135bfda-proxy-tls\") pod \"splitter-graph-78161-df5b9dc88-hxgtn\" (UID: \"fe6dbce6-a06a-4c6c-a910-0dd83135bfda\") " pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"
Sep 09 21:23:55.696774 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.696756    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe6dbce6-a06a-4c6c-a910-0dd83135bfda-proxy-tls\") pod \"splitter-graph-78161-df5b9dc88-hxgtn\" (UID: \"fe6dbce6-a06a-4c6c-a910-0dd83135bfda\") " pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"
Sep 09 21:23:55.926655 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:55.926606    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"
Sep 09 21:23:56.204635 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:56.204611    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"]
Sep 09 21:23:56.206768 ip-10-0-129-24 kubenswrapper[2585]: W0909 21:23:56.206732    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe6dbce6_a06a_4c6c_a910_0dd83135bfda.slice/crio-357504dfe1cac86f55818c0208c90c7c673ccd978597af20d88abf041be2531f WatchSource:0}: Error finding container 357504dfe1cac86f55818c0208c90c7c673ccd978597af20d88abf041be2531f: Status 404 returned error can't find the container with id 357504dfe1cac86f55818c0208c90c7c673ccd978597af20d88abf041be2531f
Sep 09 21:23:56.799055 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:56.799017    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c" path="/var/lib/kubelet/pods/30c2e7f3-ae8a-4dae-8f2c-4bbcee274d4c/volumes"
Sep 09 21:23:57.062444 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:57.062360    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn" event={"ID":"fe6dbce6-a06a-4c6c-a910-0dd83135bfda","Type":"ContainerStarted","Data":"d1eff8caa7d67bc82de3f277cf9ece8e4f6f91e867a5278f002b7ca385ae5768"}
Sep 09 21:23:57.062444 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:57.062410    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn" event={"ID":"fe6dbce6-a06a-4c6c-a910-0dd83135bfda","Type":"ContainerStarted","Data":"357504dfe1cac86f55818c0208c90c7c673ccd978597af20d88abf041be2531f"}
Sep 09 21:23:57.062842 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:57.062483    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"
Sep 09 21:23:57.081820 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:23:57.081647    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn" podStartSLOduration=3.081626535 podStartE2EDuration="3.081626535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:23:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:23:57.081068724 +0000 UTC m=+1888.751698423" watchObservedRunningTime="2026-09-09 21:23:57.081626535 +0000 UTC m=+1888.752256217"
Sep 09 21:24:03.072566 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:03.072539    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"
Sep 09 21:24:05.081519 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:05.081488    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"]
Sep 09 21:24:05.081913 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:05.081667    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn" podUID="fe6dbce6-a06a-4c6c-a910-0dd83135bfda" containerName="splitter-graph-78161" containerID="cri-o://d1eff8caa7d67bc82de3f277cf9ece8e4f6f91e867a5278f002b7ca385ae5768" gracePeriod=30
Sep 09 21:24:08.071010 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:08.070970    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn" podUID="fe6dbce6-a06a-4c6c-a910-0dd83135bfda" containerName="splitter-graph-78161" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:24:13.071380 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:13.071331    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn" podUID="fe6dbce6-a06a-4c6c-a910-0dd83135bfda" containerName="splitter-graph-78161" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:24:18.071611 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:18.071567    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn" podUID="fe6dbce6-a06a-4c6c-a910-0dd83135bfda" containerName="splitter-graph-78161" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:24:18.072016 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:18.071673    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"
Sep 09 21:24:23.071837 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:23.071795    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn" podUID="fe6dbce6-a06a-4c6c-a910-0dd83135bfda" containerName="splitter-graph-78161" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:24:24.388227 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:24.388193    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"]
Sep 09 21:24:24.391373 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:24.391348    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"
Sep 09 21:24:24.395957 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:24.395936    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-72d00-serving-cert\""
Sep 09 21:24:24.396635 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:24.396603    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-72d00-kube-rbac-proxy-sar-config\""
Sep 09 21:24:24.406051 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:24.406031    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"]
Sep 09 21:24:24.502431 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:24.502398    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850567f8-b18f-477c-8f8c-b3f7098c8000-openshift-service-ca-bundle\") pod \"switch-graph-72d00-577f44b55d-lpbf6\" (UID: \"850567f8-b18f-477c-8f8c-b3f7098c8000\") " pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"
Sep 09 21:24:24.502431 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:24.502433    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/850567f8-b18f-477c-8f8c-b3f7098c8000-proxy-tls\") pod \"switch-graph-72d00-577f44b55d-lpbf6\" (UID: \"850567f8-b18f-477c-8f8c-b3f7098c8000\") " pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"
Sep 09 21:24:24.603366 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:24.603322    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850567f8-b18f-477c-8f8c-b3f7098c8000-openshift-service-ca-bundle\") pod \"switch-graph-72d00-577f44b55d-lpbf6\" (UID: \"850567f8-b18f-477c-8f8c-b3f7098c8000\") " pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"
Sep 09 21:24:24.603366 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:24.603366    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/850567f8-b18f-477c-8f8c-b3f7098c8000-proxy-tls\") pod \"switch-graph-72d00-577f44b55d-lpbf6\" (UID: \"850567f8-b18f-477c-8f8c-b3f7098c8000\") " pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"
Sep 09 21:24:24.603570 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:24:24.603500    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-72d00-serving-cert: secret "switch-graph-72d00-serving-cert" not found
Sep 09 21:24:24.603608 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:24:24.603579    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/850567f8-b18f-477c-8f8c-b3f7098c8000-proxy-tls podName:850567f8-b18f-477c-8f8c-b3f7098c8000 nodeName:}" failed. No retries permitted until 2026-09-09 21:24:25.103563092 +0000 UTC m=+1916.774192771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/850567f8-b18f-477c-8f8c-b3f7098c8000-proxy-tls") pod "switch-graph-72d00-577f44b55d-lpbf6" (UID: "850567f8-b18f-477c-8f8c-b3f7098c8000") : secret "switch-graph-72d00-serving-cert" not found
Sep 09 21:24:24.603988 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:24.603968    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850567f8-b18f-477c-8f8c-b3f7098c8000-openshift-service-ca-bundle\") pod \"switch-graph-72d00-577f44b55d-lpbf6\" (UID: \"850567f8-b18f-477c-8f8c-b3f7098c8000\") " pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"
Sep 09 21:24:25.106890 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:25.106846    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/850567f8-b18f-477c-8f8c-b3f7098c8000-proxy-tls\") pod \"switch-graph-72d00-577f44b55d-lpbf6\" (UID: \"850567f8-b18f-477c-8f8c-b3f7098c8000\") " pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"
Sep 09 21:24:25.109312 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:25.109286    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/850567f8-b18f-477c-8f8c-b3f7098c8000-proxy-tls\") pod \"switch-graph-72d00-577f44b55d-lpbf6\" (UID: \"850567f8-b18f-477c-8f8c-b3f7098c8000\") " pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"
Sep 09 21:24:25.301257 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:25.301211    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"
Sep 09 21:24:25.572859 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:25.572835    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"]
Sep 09 21:24:25.575367 ip-10-0-129-24 kubenswrapper[2585]: W0909 21:24:25.575333    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod850567f8_b18f_477c_8f8c_b3f7098c8000.slice/crio-9617c486b2bf6e2d6ab6037aab187c05c89ea00bc370348552e362c704c19eae WatchSource:0}: Error finding container 9617c486b2bf6e2d6ab6037aab187c05c89ea00bc370348552e362c704c19eae: Status 404 returned error can't find the container with id 9617c486b2bf6e2d6ab6037aab187c05c89ea00bc370348552e362c704c19eae
Sep 09 21:24:26.170473 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:26.170436    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6" event={"ID":"850567f8-b18f-477c-8f8c-b3f7098c8000","Type":"ContainerStarted","Data":"2fd80e97ed39a0c2fbdab476349c832e4c8ec8ffbecbd302ca1de82b4203bb64"}
Sep 09 21:24:26.170654 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:26.170497    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6" event={"ID":"850567f8-b18f-477c-8f8c-b3f7098c8000","Type":"ContainerStarted","Data":"9617c486b2bf6e2d6ab6037aab187c05c89ea00bc370348552e362c704c19eae"}
Sep 09 21:24:26.170654 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:26.170543    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"
Sep 09 21:24:26.193719 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:26.193665    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6" podStartSLOduration=2.193647753 podStartE2EDuration="2.193647753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:24:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:24:26.191602283 +0000 UTC m=+1917.862231984" watchObservedRunningTime="2026-09-09 21:24:26.193647753 +0000 UTC m=+1917.864277473"
Sep 09 21:24:28.071468 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:28.071424    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn" podUID="fe6dbce6-a06a-4c6c-a910-0dd83135bfda" containerName="splitter-graph-78161" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:24:32.180031 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:32.180003    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"
Sep 09 21:24:33.075819 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:33.075773    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn" podUID="fe6dbce6-a06a-4c6c-a910-0dd83135bfda" containerName="splitter-graph-78161" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:24:35.204672 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:35.204634    2585 generic.go:356] "Generic (PLEG): container finished" podID="fe6dbce6-a06a-4c6c-a910-0dd83135bfda" containerID="d1eff8caa7d67bc82de3f277cf9ece8e4f6f91e867a5278f002b7ca385ae5768" exitCode=0
Sep 09 21:24:35.205192 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:35.204711    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn" event={"ID":"fe6dbce6-a06a-4c6c-a910-0dd83135bfda","Type":"ContainerDied","Data":"d1eff8caa7d67bc82de3f277cf9ece8e4f6f91e867a5278f002b7ca385ae5768"}
Sep 09 21:24:35.228207 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:35.228183    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"
Sep 09 21:24:35.282736 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:35.282704    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe6dbce6-a06a-4c6c-a910-0dd83135bfda-proxy-tls\") pod \"fe6dbce6-a06a-4c6c-a910-0dd83135bfda\" (UID: \"fe6dbce6-a06a-4c6c-a910-0dd83135bfda\") "
Sep 09 21:24:35.282933 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:35.282760    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe6dbce6-a06a-4c6c-a910-0dd83135bfda-openshift-service-ca-bundle\") pod \"fe6dbce6-a06a-4c6c-a910-0dd83135bfda\" (UID: \"fe6dbce6-a06a-4c6c-a910-0dd83135bfda\") "
Sep 09 21:24:35.283173 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:35.283146    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe6dbce6-a06a-4c6c-a910-0dd83135bfda-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "fe6dbce6-a06a-4c6c-a910-0dd83135bfda" (UID: "fe6dbce6-a06a-4c6c-a910-0dd83135bfda"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:24:35.284843 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:35.284813    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe6dbce6-a06a-4c6c-a910-0dd83135bfda-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "fe6dbce6-a06a-4c6c-a910-0dd83135bfda" (UID: "fe6dbce6-a06a-4c6c-a910-0dd83135bfda"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:24:35.383759 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:35.383668    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe6dbce6-a06a-4c6c-a910-0dd83135bfda-proxy-tls\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:24:35.383759 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:35.383701    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe6dbce6-a06a-4c6c-a910-0dd83135bfda-openshift-service-ca-bundle\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:24:36.210035 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:36.210003    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"
Sep 09 21:24:36.210035 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:36.210011    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn" event={"ID":"fe6dbce6-a06a-4c6c-a910-0dd83135bfda","Type":"ContainerDied","Data":"357504dfe1cac86f55818c0208c90c7c673ccd978597af20d88abf041be2531f"}
Sep 09 21:24:36.210504 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:36.210066    2585 scope.go:117] "RemoveContainer" containerID="d1eff8caa7d67bc82de3f277cf9ece8e4f6f91e867a5278f002b7ca385ae5768"
Sep 09 21:24:36.235232 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:36.235178    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"]
Sep 09 21:24:36.236537 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:36.236517    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-78161-df5b9dc88-hxgtn"]
Sep 09 21:24:36.798758 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:24:36.798724    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe6dbce6-a06a-4c6c-a910-0dd83135bfda" path="/var/lib/kubelet/pods/fe6dbce6-a06a-4c6c-a910-0dd83135bfda/volumes"
Sep 09 21:25:05.357099 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:05.357058    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"]
Sep 09 21:25:05.357499 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:05.357391    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe6dbce6-a06a-4c6c-a910-0dd83135bfda" containerName="splitter-graph-78161"
Sep 09 21:25:05.357499 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:05.357402    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe6dbce6-a06a-4c6c-a910-0dd83135bfda" containerName="splitter-graph-78161"
Sep 09 21:25:05.357499 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:05.357455    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe6dbce6-a06a-4c6c-a910-0dd83135bfda" containerName="splitter-graph-78161"
Sep 09 21:25:05.360448 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:05.360416    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"
Sep 09 21:25:05.364106 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:05.364078    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-4085c-serving-cert\""
Sep 09 21:25:05.364245 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:05.364079    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-4085c-kube-rbac-proxy-sar-config\""
Sep 09 21:25:05.376603 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:05.376574    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"]
Sep 09 21:25:05.521732 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:05.521696    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72ba6d92-08de-4bb9-9df4-6c79fcf16026-openshift-service-ca-bundle\") pod \"splitter-graph-4085c-6d4fb848b4-7t4zl\" (UID: \"72ba6d92-08de-4bb9-9df4-6c79fcf16026\") " pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"
Sep 09 21:25:05.521935 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:05.521748    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/72ba6d92-08de-4bb9-9df4-6c79fcf16026-proxy-tls\") pod \"splitter-graph-4085c-6d4fb848b4-7t4zl\" (UID: \"72ba6d92-08de-4bb9-9df4-6c79fcf16026\") " pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"
Sep 09 21:25:05.622220 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:05.622131    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72ba6d92-08de-4bb9-9df4-6c79fcf16026-openshift-service-ca-bundle\") pod \"splitter-graph-4085c-6d4fb848b4-7t4zl\" (UID: \"72ba6d92-08de-4bb9-9df4-6c79fcf16026\") " pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"
Sep 09 21:25:05.622220 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:05.622182    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/72ba6d92-08de-4bb9-9df4-6c79fcf16026-proxy-tls\") pod \"splitter-graph-4085c-6d4fb848b4-7t4zl\" (UID: \"72ba6d92-08de-4bb9-9df4-6c79fcf16026\") " pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"
Sep 09 21:25:05.622401 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:25:05.622295    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-4085c-serving-cert: secret "splitter-graph-4085c-serving-cert" not found
Sep 09 21:25:05.622401 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:25:05.622379    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72ba6d92-08de-4bb9-9df4-6c79fcf16026-proxy-tls podName:72ba6d92-08de-4bb9-9df4-6c79fcf16026 nodeName:}" failed. No retries permitted until 2026-09-09 21:25:06.122360466 +0000 UTC m=+1957.792990147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/72ba6d92-08de-4bb9-9df4-6c79fcf16026-proxy-tls") pod "splitter-graph-4085c-6d4fb848b4-7t4zl" (UID: "72ba6d92-08de-4bb9-9df4-6c79fcf16026") : secret "splitter-graph-4085c-serving-cert" not found
Sep 09 21:25:05.622754 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:05.622730    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72ba6d92-08de-4bb9-9df4-6c79fcf16026-openshift-service-ca-bundle\") pod \"splitter-graph-4085c-6d4fb848b4-7t4zl\" (UID: \"72ba6d92-08de-4bb9-9df4-6c79fcf16026\") " pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"
Sep 09 21:25:06.127805 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:06.127770    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/72ba6d92-08de-4bb9-9df4-6c79fcf16026-proxy-tls\") pod \"splitter-graph-4085c-6d4fb848b4-7t4zl\" (UID: \"72ba6d92-08de-4bb9-9df4-6c79fcf16026\") " pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"
Sep 09 21:25:06.130330 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:06.130302    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/72ba6d92-08de-4bb9-9df4-6c79fcf16026-proxy-tls\") pod \"splitter-graph-4085c-6d4fb848b4-7t4zl\" (UID: \"72ba6d92-08de-4bb9-9df4-6c79fcf16026\") " pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"
Sep 09 21:25:06.270869 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:06.270832    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"
Sep 09 21:25:06.562339 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:06.562315    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"]
Sep 09 21:25:06.564468 ip-10-0-129-24 kubenswrapper[2585]: W0909 21:25:06.564439    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72ba6d92_08de_4bb9_9df4_6c79fcf16026.slice/crio-898308c2944ab10e9b19397a39bfe06fb61b5464507a6d9036342f4a36f537f5 WatchSource:0}: Error finding container 898308c2944ab10e9b19397a39bfe06fb61b5464507a6d9036342f4a36f537f5: Status 404 returned error can't find the container with id 898308c2944ab10e9b19397a39bfe06fb61b5464507a6d9036342f4a36f537f5
Sep 09 21:25:07.327087 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:07.327050    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl" event={"ID":"72ba6d92-08de-4bb9-9df4-6c79fcf16026","Type":"ContainerStarted","Data":"719639ea8f2a2667fc6423e9915db36f883d0aa54adc797183ee90118020a933"}
Sep 09 21:25:07.327288 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:07.327101    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl" event={"ID":"72ba6d92-08de-4bb9-9df4-6c79fcf16026","Type":"ContainerStarted","Data":"898308c2944ab10e9b19397a39bfe06fb61b5464507a6d9036342f4a36f537f5"}
Sep 09 21:25:07.327288 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:07.327136    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"
Sep 09 21:25:07.349513 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:07.349459    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl" podStartSLOduration=2.349441484 podStartE2EDuration="2.349441484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:25:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:25:07.348143168 +0000 UTC m=+1959.018772870" watchObservedRunningTime="2026-09-09 21:25:07.349441484 +0000 UTC m=+1959.020071183"
Sep 09 21:25:13.336454 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:25:13.336421    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"
Sep 09 21:33:19.927265 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:19.927228    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"]
Sep 09 21:33:19.927850 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:19.927504    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl" podUID="72ba6d92-08de-4bb9-9df4-6c79fcf16026" containerName="splitter-graph-4085c" containerID="cri-o://719639ea8f2a2667fc6423e9915db36f883d0aa54adc797183ee90118020a933" gracePeriod=30
Sep 09 21:33:23.334793 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:23.334753    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl" podUID="72ba6d92-08de-4bb9-9df4-6c79fcf16026" containerName="splitter-graph-4085c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:33:28.334646 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:28.334602    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl" podUID="72ba6d92-08de-4bb9-9df4-6c79fcf16026" containerName="splitter-graph-4085c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:33:33.334537 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:33.334499    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl" podUID="72ba6d92-08de-4bb9-9df4-6c79fcf16026" containerName="splitter-graph-4085c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:33:33.334927 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:33.334595    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"
Sep 09 21:33:38.334601 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:38.334563    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl" podUID="72ba6d92-08de-4bb9-9df4-6c79fcf16026" containerName="splitter-graph-4085c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:33:43.334727 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:43.334687    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl" podUID="72ba6d92-08de-4bb9-9df4-6c79fcf16026" containerName="splitter-graph-4085c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:33:48.334201 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:48.334161    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl" podUID="72ba6d92-08de-4bb9-9df4-6c79fcf16026" containerName="splitter-graph-4085c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:33:50.071546 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.071518    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"
Sep 09 21:33:50.202345 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.202259    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/72ba6d92-08de-4bb9-9df4-6c79fcf16026-proxy-tls\") pod \"72ba6d92-08de-4bb9-9df4-6c79fcf16026\" (UID: \"72ba6d92-08de-4bb9-9df4-6c79fcf16026\") "
Sep 09 21:33:50.202345 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.202341    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72ba6d92-08de-4bb9-9df4-6c79fcf16026-openshift-service-ca-bundle\") pod \"72ba6d92-08de-4bb9-9df4-6c79fcf16026\" (UID: \"72ba6d92-08de-4bb9-9df4-6c79fcf16026\") "
Sep 09 21:33:50.202729 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.202703    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72ba6d92-08de-4bb9-9df4-6c79fcf16026-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "72ba6d92-08de-4bb9-9df4-6c79fcf16026" (UID: "72ba6d92-08de-4bb9-9df4-6c79fcf16026"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:33:50.204352 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.204323    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72ba6d92-08de-4bb9-9df4-6c79fcf16026-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "72ba6d92-08de-4bb9-9df4-6c79fcf16026" (UID: "72ba6d92-08de-4bb9-9df4-6c79fcf16026"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:33:50.249763 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.249728    2585 generic.go:356] "Generic (PLEG): container finished" podID="72ba6d92-08de-4bb9-9df4-6c79fcf16026" containerID="719639ea8f2a2667fc6423e9915db36f883d0aa54adc797183ee90118020a933" exitCode=0
Sep 09 21:33:50.249948 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.249774    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"
Sep 09 21:33:50.249948 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.249780    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl" event={"ID":"72ba6d92-08de-4bb9-9df4-6c79fcf16026","Type":"ContainerDied","Data":"719639ea8f2a2667fc6423e9915db36f883d0aa54adc797183ee90118020a933"}
Sep 09 21:33:50.249948 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.249906    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl" event={"ID":"72ba6d92-08de-4bb9-9df4-6c79fcf16026","Type":"ContainerDied","Data":"898308c2944ab10e9b19397a39bfe06fb61b5464507a6d9036342f4a36f537f5"}
Sep 09 21:33:50.250054 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.249958    2585 scope.go:117] "RemoveContainer" containerID="719639ea8f2a2667fc6423e9915db36f883d0aa54adc797183ee90118020a933"
Sep 09 21:33:50.258791 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.258769    2585 scope.go:117] "RemoveContainer" containerID="719639ea8f2a2667fc6423e9915db36f883d0aa54adc797183ee90118020a933"
Sep 09 21:33:50.259133 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:33:50.259113    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"719639ea8f2a2667fc6423e9915db36f883d0aa54adc797183ee90118020a933\": container with ID starting with 719639ea8f2a2667fc6423e9915db36f883d0aa54adc797183ee90118020a933 not found: ID does not exist" containerID="719639ea8f2a2667fc6423e9915db36f883d0aa54adc797183ee90118020a933"
Sep 09 21:33:50.259205 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.259142    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"719639ea8f2a2667fc6423e9915db36f883d0aa54adc797183ee90118020a933"} err="failed to get container status \"719639ea8f2a2667fc6423e9915db36f883d0aa54adc797183ee90118020a933\": rpc error: code = NotFound desc = could not find container \"719639ea8f2a2667fc6423e9915db36f883d0aa54adc797183ee90118020a933\": container with ID starting with 719639ea8f2a2667fc6423e9915db36f883d0aa54adc797183ee90118020a933 not found: ID does not exist"
Sep 09 21:33:50.273635 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.273599    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"]
Sep 09 21:33:50.278280 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.278253    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4085c-6d4fb848b4-7t4zl"]
Sep 09 21:33:50.303237 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.303205    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/72ba6d92-08de-4bb9-9df4-6c79fcf16026-proxy-tls\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:33:50.303237 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.303236    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72ba6d92-08de-4bb9-9df4-6c79fcf16026-openshift-service-ca-bundle\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:33:50.798686 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:33:50.798640    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72ba6d92-08de-4bb9-9df4-6c79fcf16026" path="/var/lib/kubelet/pods/72ba6d92-08de-4bb9-9df4-6c79fcf16026/volumes"
Sep 09 21:40:43.632747 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:40:43.632662    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"]
Sep 09 21:40:43.635165 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:40:43.632869    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6" podUID="850567f8-b18f-477c-8f8c-b3f7098c8000" containerName="switch-graph-72d00" containerID="cri-o://2fd80e97ed39a0c2fbdab476349c832e4c8ec8ffbecbd302ca1de82b4203bb64" gracePeriod=30
Sep 09 21:40:47.178540 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:40:47.178500    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6" podUID="850567f8-b18f-477c-8f8c-b3f7098c8000" containerName="switch-graph-72d00" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:40:52.178373 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:40:52.178330    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6" podUID="850567f8-b18f-477c-8f8c-b3f7098c8000" containerName="switch-graph-72d00" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:40:57.178225 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:40:57.178173    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6" podUID="850567f8-b18f-477c-8f8c-b3f7098c8000" containerName="switch-graph-72d00" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:40:57.178636 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:40:57.178278    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"
Sep 09 21:41:00.209770 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:00.209739    2585 ???:1] "http: TLS handshake error from 10.0.140.67:41962: EOF"
Sep 09 21:41:00.212411 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:00.212384    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-72d00-577f44b55d-lpbf6_850567f8-b18f-477c-8f8c-b3f7098c8000/switch-graph-72d00/0.log"
Sep 09 21:41:01.096577 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:01.096535    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-72d00-577f44b55d-lpbf6_850567f8-b18f-477c-8f8c-b3f7098c8000/switch-graph-72d00/0.log"
Sep 09 21:41:01.957466 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:01.957437    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-72d00-577f44b55d-lpbf6_850567f8-b18f-477c-8f8c-b3f7098c8000/switch-graph-72d00/0.log"
Sep 09 21:41:02.178081 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:02.178036    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6" podUID="850567f8-b18f-477c-8f8c-b3f7098c8000" containerName="switch-graph-72d00" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:41:02.748386 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:02.748334    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-72d00-577f44b55d-lpbf6_850567f8-b18f-477c-8f8c-b3f7098c8000/switch-graph-72d00/0.log"
Sep 09 21:41:03.540144 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:03.540107    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-72d00-577f44b55d-lpbf6_850567f8-b18f-477c-8f8c-b3f7098c8000/switch-graph-72d00/0.log"
Sep 09 21:41:04.341111 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:04.341083    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-72d00-577f44b55d-lpbf6_850567f8-b18f-477c-8f8c-b3f7098c8000/switch-graph-72d00/0.log"
Sep 09 21:41:05.193463 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:05.193428    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-72d00-577f44b55d-lpbf6_850567f8-b18f-477c-8f8c-b3f7098c8000/switch-graph-72d00/0.log"
Sep 09 21:41:05.977356 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:05.977325    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-72d00-577f44b55d-lpbf6_850567f8-b18f-477c-8f8c-b3f7098c8000/switch-graph-72d00/0.log"
Sep 09 21:41:06.754852 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:06.754816    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-72d00-577f44b55d-lpbf6_850567f8-b18f-477c-8f8c-b3f7098c8000/switch-graph-72d00/0.log"
Sep 09 21:41:07.178293 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:07.178241    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6" podUID="850567f8-b18f-477c-8f8c-b3f7098c8000" containerName="switch-graph-72d00" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:41:07.539082 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:07.539057    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-72d00-577f44b55d-lpbf6_850567f8-b18f-477c-8f8c-b3f7098c8000/switch-graph-72d00/0.log"
Sep 09 21:41:08.328664 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:08.328632    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-72d00-577f44b55d-lpbf6_850567f8-b18f-477c-8f8c-b3f7098c8000/switch-graph-72d00/0.log"
Sep 09 21:41:09.103222 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:09.103192    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-72d00-577f44b55d-lpbf6_850567f8-b18f-477c-8f8c-b3f7098c8000/switch-graph-72d00/0.log"
Sep 09 21:41:09.892511 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:09.892481    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-72d00-577f44b55d-lpbf6_850567f8-b18f-477c-8f8c-b3f7098c8000/switch-graph-72d00/0.log"
Sep 09 21:41:10.718287 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:10.718256    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-72d00-577f44b55d-lpbf6_850567f8-b18f-477c-8f8c-b3f7098c8000/switch-graph-72d00/0.log"
Sep 09 21:41:12.177963 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:12.177925    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6" podUID="850567f8-b18f-477c-8f8c-b3f7098c8000" containerName="switch-graph-72d00" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:41:13.769500 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:13.769472    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"
Sep 09 21:41:13.871647 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:13.871617    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/850567f8-b18f-477c-8f8c-b3f7098c8000-proxy-tls\") pod \"850567f8-b18f-477c-8f8c-b3f7098c8000\" (UID: \"850567f8-b18f-477c-8f8c-b3f7098c8000\") "
Sep 09 21:41:13.871826 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:13.871657    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850567f8-b18f-477c-8f8c-b3f7098c8000-openshift-service-ca-bundle\") pod \"850567f8-b18f-477c-8f8c-b3f7098c8000\" (UID: \"850567f8-b18f-477c-8f8c-b3f7098c8000\") "
Sep 09 21:41:13.872069 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:13.872039    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/850567f8-b18f-477c-8f8c-b3f7098c8000-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "850567f8-b18f-477c-8f8c-b3f7098c8000" (UID: "850567f8-b18f-477c-8f8c-b3f7098c8000"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:41:13.873734 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:13.873713    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/850567f8-b18f-477c-8f8c-b3f7098c8000-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "850567f8-b18f-477c-8f8c-b3f7098c8000" (UID: "850567f8-b18f-477c-8f8c-b3f7098c8000"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:41:13.885004 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:13.884981    2585 generic.go:356] "Generic (PLEG): container finished" podID="850567f8-b18f-477c-8f8c-b3f7098c8000" containerID="2fd80e97ed39a0c2fbdab476349c832e4c8ec8ffbecbd302ca1de82b4203bb64" exitCode=0
Sep 09 21:41:13.885118 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:13.885022    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"
Sep 09 21:41:13.885118 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:13.885066    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6" event={"ID":"850567f8-b18f-477c-8f8c-b3f7098c8000","Type":"ContainerDied","Data":"2fd80e97ed39a0c2fbdab476349c832e4c8ec8ffbecbd302ca1de82b4203bb64"}
Sep 09 21:41:13.885197 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:13.885124    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6" event={"ID":"850567f8-b18f-477c-8f8c-b3f7098c8000","Type":"ContainerDied","Data":"9617c486b2bf6e2d6ab6037aab187c05c89ea00bc370348552e362c704c19eae"}
Sep 09 21:41:13.885197 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:13.885150    2585 scope.go:117] "RemoveContainer" containerID="2fd80e97ed39a0c2fbdab476349c832e4c8ec8ffbecbd302ca1de82b4203bb64"
Sep 09 21:41:13.895734 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:13.895715    2585 scope.go:117] "RemoveContainer" containerID="2fd80e97ed39a0c2fbdab476349c832e4c8ec8ffbecbd302ca1de82b4203bb64"
Sep 09 21:41:13.896065 ip-10-0-129-24 kubenswrapper[2585]: E0909 21:41:13.896041    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fd80e97ed39a0c2fbdab476349c832e4c8ec8ffbecbd302ca1de82b4203bb64\": container with ID starting with 2fd80e97ed39a0c2fbdab476349c832e4c8ec8ffbecbd302ca1de82b4203bb64 not found: ID does not exist" containerID="2fd80e97ed39a0c2fbdab476349c832e4c8ec8ffbecbd302ca1de82b4203bb64"
Sep 09 21:41:13.896121 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:13.896077    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fd80e97ed39a0c2fbdab476349c832e4c8ec8ffbecbd302ca1de82b4203bb64"} err="failed to get container status \"2fd80e97ed39a0c2fbdab476349c832e4c8ec8ffbecbd302ca1de82b4203bb64\": rpc error: code = NotFound desc = could not find container \"2fd80e97ed39a0c2fbdab476349c832e4c8ec8ffbecbd302ca1de82b4203bb64\": container with ID starting with 2fd80e97ed39a0c2fbdab476349c832e4c8ec8ffbecbd302ca1de82b4203bb64 not found: ID does not exist"
Sep 09 21:41:13.907449 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:13.907423    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"]
Sep 09 21:41:13.910382 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:13.910361    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-72d00-577f44b55d-lpbf6"]
Sep 09 21:41:13.972136 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:13.972104    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/850567f8-b18f-477c-8f8c-b3f7098c8000-proxy-tls\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:41:13.972136 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:13.972130    2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/850567f8-b18f-477c-8f8c-b3f7098c8000-openshift-service-ca-bundle\") on node \"ip-10-0-129-24.ec2.internal\" DevicePath \"\""
Sep 09 21:41:14.797668 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:14.797629    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="850567f8-b18f-477c-8f8c-b3f7098c8000" path="/var/lib/kubelet/pods/850567f8-b18f-477c-8f8c-b3f7098c8000/volumes"
Sep 09 21:41:19.037627 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:19.037591    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-c28zc_95b5dc36-8d5d-43d2-9444-5e870b1ca94b/global-pull-secret-syncer/0.log"
Sep 09 21:41:19.259668 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:19.259636    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pb6hf_59d9f229-3224-476f-ae39-e8b4e039bae5/konnectivity-agent/0.log"
Sep 09 21:41:19.304285 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:19.304211    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-24.ec2.internal_4ad114d159ddf9784199c66d4e552938/haproxy/0.log"
Sep 09 21:41:23.281206 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:23.281172    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-jkb4d_af8e8d4d-b623-47d4-a5fb-9ca5fe30732c/kube-state-metrics/0.log"
Sep 09 21:41:23.311900 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:23.311854    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-jkb4d_af8e8d4d-b623-47d4-a5fb-9ca5fe30732c/kube-rbac-proxy-main/0.log"
Sep 09 21:41:23.345057 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:23.345027    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-jkb4d_af8e8d4d-b623-47d4-a5fb-9ca5fe30732c/kube-rbac-proxy-self/0.log"
Sep 09 21:41:23.502808 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:23.502773    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-nspsp_7c429400-a7e1-4059-afa6-4283be5c4966/monitoring-plugin/0.log"
Sep 09 21:41:23.740267 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:23.740191    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mrd8s_280a5f0f-cdd8-42d6-bf24-d1f6d9e73291/node-exporter/0.log"
Sep 09 21:41:23.778662 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:23.778633    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mrd8s_280a5f0f-cdd8-42d6-bf24-d1f6d9e73291/kube-rbac-proxy/0.log"
Sep 09 21:41:23.809902 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:23.809851    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mrd8s_280a5f0f-cdd8-42d6-bf24-d1f6d9e73291/init-textfile/0.log"
Sep 09 21:41:26.933421 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:26.933382    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"]
Sep 09 21:41:26.933918 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:26.933791    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72ba6d92-08de-4bb9-9df4-6c79fcf16026" containerName="splitter-graph-4085c"
Sep 09 21:41:26.933918 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:26.933808    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="72ba6d92-08de-4bb9-9df4-6c79fcf16026" containerName="splitter-graph-4085c"
Sep 09 21:41:26.933918 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:26.933823    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="850567f8-b18f-477c-8f8c-b3f7098c8000" containerName="switch-graph-72d00"
Sep 09 21:41:26.933918 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:26.933831    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="850567f8-b18f-477c-8f8c-b3f7098c8000" containerName="switch-graph-72d00"
Sep 09 21:41:26.934149 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:26.933934    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="850567f8-b18f-477c-8f8c-b3f7098c8000" containerName="switch-graph-72d00"
Sep 09 21:41:26.934149 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:26.933947    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="72ba6d92-08de-4bb9-9df4-6c79fcf16026" containerName="splitter-graph-4085c"
Sep 09 21:41:26.938297 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:26.938275    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:26.941094 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:26.941073    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kmz7m\"/\"kube-root-ca.crt\""
Sep 09 21:41:26.941213 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:26.941093    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kmz7m\"/\"openshift-service-ca.crt\""
Sep 09 21:41:26.943042 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:26.943027    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kmz7m\"/\"default-dockercfg-ss6ds\""
Sep 09 21:41:26.951913 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:26.950668    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"]
Sep 09 21:41:27.077113 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.077071    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76f4b0dd-c1ea-450e-8cf9-16bdeafb954e-lib-modules\") pod \"perf-node-gather-daemonset-nhkzm\" (UID: \"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e\") " pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.077328 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.077121    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm6n2\" (UniqueName: \"kubernetes.io/projected/76f4b0dd-c1ea-450e-8cf9-16bdeafb954e-kube-api-access-vm6n2\") pod \"perf-node-gather-daemonset-nhkzm\" (UID: \"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e\") " pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.077328 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.077234    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76f4b0dd-c1ea-450e-8cf9-16bdeafb954e-sys\") pod \"perf-node-gather-daemonset-nhkzm\" (UID: \"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e\") " pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.077328 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.077262    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/76f4b0dd-c1ea-450e-8cf9-16bdeafb954e-proc\") pod \"perf-node-gather-daemonset-nhkzm\" (UID: \"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e\") " pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.077328 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.077280    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/76f4b0dd-c1ea-450e-8cf9-16bdeafb954e-podres\") pod \"perf-node-gather-daemonset-nhkzm\" (UID: \"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e\") " pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.178549 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.178505    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76f4b0dd-c1ea-450e-8cf9-16bdeafb954e-sys\") pod \"perf-node-gather-daemonset-nhkzm\" (UID: \"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e\") " pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.178549 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.178546    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/76f4b0dd-c1ea-450e-8cf9-16bdeafb954e-proc\") pod \"perf-node-gather-daemonset-nhkzm\" (UID: \"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e\") " pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.178807 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.178569    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/76f4b0dd-c1ea-450e-8cf9-16bdeafb954e-podres\") pod \"perf-node-gather-daemonset-nhkzm\" (UID: \"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e\") " pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.178807 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.178609    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76f4b0dd-c1ea-450e-8cf9-16bdeafb954e-lib-modules\") pod \"perf-node-gather-daemonset-nhkzm\" (UID: \"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e\") " pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.178807 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.178628    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm6n2\" (UniqueName: \"kubernetes.io/projected/76f4b0dd-c1ea-450e-8cf9-16bdeafb954e-kube-api-access-vm6n2\") pod \"perf-node-gather-daemonset-nhkzm\" (UID: \"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e\") " pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.178807 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.178670    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76f4b0dd-c1ea-450e-8cf9-16bdeafb954e-sys\") pod \"perf-node-gather-daemonset-nhkzm\" (UID: \"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e\") " pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.178807 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.178743    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/76f4b0dd-c1ea-450e-8cf9-16bdeafb954e-proc\") pod \"perf-node-gather-daemonset-nhkzm\" (UID: \"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e\") " pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.178807 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.178747    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/76f4b0dd-c1ea-450e-8cf9-16bdeafb954e-podres\") pod \"perf-node-gather-daemonset-nhkzm\" (UID: \"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e\") " pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.178807 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.178769    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76f4b0dd-c1ea-450e-8cf9-16bdeafb954e-lib-modules\") pod \"perf-node-gather-daemonset-nhkzm\" (UID: \"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e\") " pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.187620 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.187549    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm6n2\" (UniqueName: \"kubernetes.io/projected/76f4b0dd-c1ea-450e-8cf9-16bdeafb954e-kube-api-access-vm6n2\") pod \"perf-node-gather-daemonset-nhkzm\" (UID: \"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e\") " pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.251697 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.251658    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.525206 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.525120    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"]
Sep 09 21:41:27.528365 ip-10-0-129-24 kubenswrapper[2585]: W0909 21:41:27.528322    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod76f4b0dd_c1ea_450e_8cf9_16bdeafb954e.slice/crio-5f436477a45542e0735b6be5fe8d92973feaa4b827a5e0d5e993fb443613d1eb WatchSource:0}: Error finding container 5f436477a45542e0735b6be5fe8d92973feaa4b827a5e0d5e993fb443613d1eb: Status 404 returned error can't find the container with id 5f436477a45542e0735b6be5fe8d92973feaa4b827a5e0d5e993fb443613d1eb
Sep 09 21:41:27.936344 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.936306    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm" event={"ID":"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e","Type":"ContainerStarted","Data":"ab709563aa42f5f48119c887b837393aaf4fb2601261c0832af866599b26ef22"}
Sep 09 21:41:27.936730 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.936367    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm" event={"ID":"76f4b0dd-c1ea-450e-8cf9-16bdeafb954e","Type":"ContainerStarted","Data":"5f436477a45542e0735b6be5fe8d92973feaa4b827a5e0d5e993fb443613d1eb"}
Sep 09 21:41:27.936730 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.936421    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:27.955786 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:27.955736    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm" podStartSLOduration=1.955719494 podStartE2EDuration="1.955719494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:41:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:41:27.954577528 +0000 UTC m=+2939.625207229" watchObservedRunningTime="2026-09-09 21:41:27.955719494 +0000 UTC m=+2939.626349195"
Sep 09 21:41:28.672307 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:28.672265    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gc9kl_e74c0d43-20df-4f97-9b0f-654dd80de9c3/dns/0.log"
Sep 09 21:41:28.696200 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:28.696169    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gc9kl_e74c0d43-20df-4f97-9b0f-654dd80de9c3/kube-rbac-proxy/0.log"
Sep 09 21:41:28.768120 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:28.768087    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gvwv2_d23607fd-429c-4a38-acc1-fbac3cd34147/dns-node-resolver/0.log"
Sep 09 21:41:29.335658 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:29.335628    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-d84dd5b4f-mjmz8_a82a19d7-0128-4d70-a667-370ad882f303/registry/0.log"
Sep 09 21:41:29.408440 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:29.408416    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kgvvq_3f399b85-c8bd-417f-ae6a-1085b82a4a58/node-ca/0.log"
Sep 09 21:41:30.542780 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:30.542755    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6bcgn_19c41e9b-65f1-4408-be09-392b760bd6a5/serve-healthcheck-canary/0.log"
Sep 09 21:41:31.095858 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:31.095828    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kzn2d_befc00a5-d23a-41b4-b476-af36f0ed0cd9/kube-rbac-proxy/0.log"
Sep 09 21:41:31.118769 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:31.118741    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kzn2d_befc00a5-d23a-41b4-b476-af36f0ed0cd9/exporter/0.log"
Sep 09 21:41:31.142297 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:31.142270    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kzn2d_befc00a5-d23a-41b4-b476-af36f0ed0cd9/extractor/0.log"
Sep 09 21:41:33.416663 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:33.416630    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-644949856f-8wqzq_478dd089-0d75-441b-be7a-015b3b91af57/manager/0.log"
Sep 09 21:41:33.470265 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:33.470240    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-7b954ffc95-56488_9e94f480-d81c-4edc-9fad-7fb1dda28645/server/0.log"
Sep 09 21:41:33.553315 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:33.553284    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-7b966d7984-phpqj_beaff307-ce71-4909-95f5-ba702a791992/manager/0.log"
Sep 09 21:41:33.643205 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:33.643175    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-kw4nd_ff1febf7-4e22-4442-a4ef-9ddb04ede5be/seaweedfs/0.log"
Sep 09 21:41:33.949536 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:33.949507    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kmz7m/perf-node-gather-daemonset-nhkzm"
Sep 09 21:41:37.924548 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:37.924520    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-vk9h4_adb45051-79cf-498b-81b4-5f483067b937/migrator/0.log"
Sep 09 21:41:37.963187 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:37.963159    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-vk9h4_adb45051-79cf-498b-81b4-5f483067b937/graceful-termination/0.log"
Sep 09 21:41:39.202956 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:39.202931    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pwq6l_3f665bfb-5209-4095-9276-31d40e0d326b/kube-multus-additional-cni-plugins/0.log"
Sep 09 21:41:39.257326 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:39.257298    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pwq6l_3f665bfb-5209-4095-9276-31d40e0d326b/egress-router-binary-copy/0.log"
Sep 09 21:41:39.310627 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:39.310600    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pwq6l_3f665bfb-5209-4095-9276-31d40e0d326b/cni-plugins/0.log"
Sep 09 21:41:39.334160 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:39.334135    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pwq6l_3f665bfb-5209-4095-9276-31d40e0d326b/bond-cni-plugin/0.log"
Sep 09 21:41:39.359382 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:39.359357    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pwq6l_3f665bfb-5209-4095-9276-31d40e0d326b/routeoverride-cni/0.log"
Sep 09 21:41:39.385716 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:39.385685    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pwq6l_3f665bfb-5209-4095-9276-31d40e0d326b/whereabouts-cni-bincopy/0.log"
Sep 09 21:41:39.410244 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:39.410213    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pwq6l_3f665bfb-5209-4095-9276-31d40e0d326b/whereabouts-cni/0.log"
Sep 09 21:41:39.838958 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:39.838924    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-x5c85_1ffdae73-5e32-418c-8138-171b99a0c118/kube-multus/0.log"
Sep 09 21:41:39.954352 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:39.954326    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ltd4k_c907cd72-cf97-4133-87f8-2e9bb7fb67a6/network-metrics-daemon/0.log"
Sep 09 21:41:39.976768 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:39.976741    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ltd4k_c907cd72-cf97-4133-87f8-2e9bb7fb67a6/kube-rbac-proxy/0.log"
Sep 09 21:41:40.775928 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:40.775896    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-82b4g_9c01881d-abb4-462f-8807-43242a0e0dbd/ovn-controller/0.log"
Sep 09 21:41:40.794891 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:40.794847    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-82b4g_9c01881d-abb4-462f-8807-43242a0e0dbd/ovn-acl-logging/0.log"
Sep 09 21:41:40.819147 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:40.819122    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-82b4g_9c01881d-abb4-462f-8807-43242a0e0dbd/kube-rbac-proxy-node/0.log"
Sep 09 21:41:40.841193 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:40.841146    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-82b4g_9c01881d-abb4-462f-8807-43242a0e0dbd/kube-rbac-proxy-ovn-metrics/0.log"
Sep 09 21:41:40.862249 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:40.862221    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-82b4g_9c01881d-abb4-462f-8807-43242a0e0dbd/northd/0.log"
Sep 09 21:41:40.885624 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:40.885601    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-82b4g_9c01881d-abb4-462f-8807-43242a0e0dbd/nbdb/0.log"
Sep 09 21:41:40.907752 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:40.907727    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-82b4g_9c01881d-abb4-462f-8807-43242a0e0dbd/sbdb/0.log"
Sep 09 21:41:40.950063 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:40.950034    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-82b4g_9c01881d-abb4-462f-8807-43242a0e0dbd/ovnkube-controller/0.log"
Sep 09 21:41:42.356295 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:42.356263    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-4ct29_04fa9846-4123-477d-8c86-481d721ecc1e/network-check-target-container/0.log"
Sep 09 21:41:43.408432 ip-10-0-129-24 kubenswrapper[2585]: I0909 21:41:43.408402    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-9pczp_0c1af91b-e37d-4a36-a741-21f2bcb399ac/iptables-alerter/0.log"