May 21 17:42:39.757109 ip-10-0-135-125 systemd[1]: Starting Kubernetes Kubelet...
May 21 17:42:40.252528 ip-10-0-135-125 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 21 17:42:40.252528 ip-10-0-135-125 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 21 17:42:40.252528 ip-10-0-135-125 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 21 17:42:40.252528 ip-10-0-135-125 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 21 17:42:40.252528 ip-10-0-135-125 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 21 17:42:40.255076 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.254976    2558 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 21 17:42:40.258771 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258751    2558 feature_gate.go:328] unrecognized feature gate: Example2
May 21 17:42:40.258771 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258768    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 21 17:42:40.258771 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258776    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258797    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258802    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258806    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258809    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258813    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258817    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258821    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258825    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258830    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258833    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258837    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258841    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258845    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258850    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258854    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258859    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258863    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258867    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 21 17:42:40.258979 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258871    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258875    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258879    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258884    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258888    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258891    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258896    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258899    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258904    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258908    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258912    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258916    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258923    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258931    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258935    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258941    2558 feature_gate.go:328] unrecognized feature gate: Example
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258946    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258951    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258955    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258960    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 21 17:42:40.259717 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258965    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258971    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258976    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258980    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258984    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258989    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258994    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.258998    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259003    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259007    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259011    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259015    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259020    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259024    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259028    2558 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259032    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259036    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259041    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259045    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259049    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 21 17:42:40.260557 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259053    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259057    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259062    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259066    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259071    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259075    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259081    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259087    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259092    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259096    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259101    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259106    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259110    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259114    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259121    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259125    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259135    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259140    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259144    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259148    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 21 17:42:40.261197 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259153    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259157    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259161    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259165    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.259169    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261071    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261089    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261093    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261097    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261100    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261106    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261112    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261118    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261123    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261126    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261129    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261132    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261136    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261139    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 21 17:42:40.261680 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261141    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261144    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261147    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261150    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261153    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261156    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261159    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261162    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261165    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261173    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261176    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261179    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261182    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261185    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261187    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261191    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261194    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261196    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261199    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 21 17:42:40.262161 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261202    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261205    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261208    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261211    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261214    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261217    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261219    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261222    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261225    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261227    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261230    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261232    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261235    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261237    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261240    2558 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261242    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261245    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261247    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261250    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261253    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 21 17:42:40.262636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261256    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261259    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261262    2558 feature_gate.go:328] unrecognized feature gate: Example
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261264    2558 feature_gate.go:328] unrecognized feature gate: Example2
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261267    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261270    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261272    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261275    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261277    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261280    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261283    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261286    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261289    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261292    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261295    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261297    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261300    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261303    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261305    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261308    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 21 17:42:40.263172 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261310    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261313    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261316    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261318    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261321    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261323    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261326    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261328    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261331    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261334    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261336    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261338    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.261342    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261415    2558 flags.go:64] FLAG: --address="0.0.0.0"
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261423    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261432    2558 flags.go:64] FLAG: --anonymous-auth="true"
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261438    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261445    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261449    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261453    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261458    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 21 17:42:40.263682 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261461    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261464    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261468    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261471    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261475    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261477    2558 flags.go:64] FLAG: --cgroup-root=""
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261481    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261484    2558 flags.go:64] FLAG: --client-ca-file=""
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261487    2558 flags.go:64] FLAG: --cloud-config=""
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261490    2558 flags.go:64] FLAG: --cloud-provider="external"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261492    2558 flags.go:64] FLAG: --cluster-dns="[]"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261498    2558 flags.go:64] FLAG: --cluster-domain=""
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261501    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261504    2558 flags.go:64] FLAG: --config-dir=""
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261506    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261510    2558 flags.go:64] FLAG: --container-log-max-files="5"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261517    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261527    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261531    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261534    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261537    2558 flags.go:64] FLAG: --contention-profiling="false"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261540    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261543    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261547    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261550    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 21 17:42:40.264287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261554    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261557    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261560    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261563    2558 flags.go:64] FLAG: --enable-load-reader="false"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261566    2558 flags.go:64] FLAG: --enable-server="true"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261569    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261574    2558 flags.go:64] FLAG: --event-burst="100"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261577    2558 flags.go:64] FLAG: --event-qps="50"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261580    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261583    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261587    2558 flags.go:64] FLAG: --eviction-hard=""
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261591    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261594    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261597    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261600    2558 flags.go:64] FLAG: --eviction-soft=""
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261603    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261606    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261609    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261612    2558 flags.go:64] FLAG: --experimental-mounter-path=""
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261615    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261618    2558 flags.go:64] FLAG: --fail-swap-on="true"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261621    2558 flags.go:64] FLAG: --feature-gates=""
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261625    2558 flags.go:64] FLAG: --file-check-frequency="20s"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261628    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261631    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 21 17:42:40.264900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261643    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261646    2558 flags.go:64] FLAG: --healthz-port="10248"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261649    2558 flags.go:64] FLAG: --help="false"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261652    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-135-125.ec2.internal"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261655    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261661    2558 flags.go:64] FLAG: --http-check-frequency="20s"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261664    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261668    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261671    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261674    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261677    2558 flags.go:64] FLAG: --image-service-endpoint=""
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261680    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261683    2558 flags.go:64] FLAG: --kube-api-burst="100"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261687    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261690    2558 flags.go:64] FLAG: --kube-api-qps="50"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261693    2558 flags.go:64] FLAG: --kube-reserved=""
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261696    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261700    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261703    2558 flags.go:64] FLAG: --kubelet-cgroups=""
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261706    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261709    2558 flags.go:64] FLAG: --lock-file=""
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261712    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261715    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261718    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 21 17:42:40.265497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261724    2558 flags.go:64] FLAG: --log-json-split-stream="false"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261727    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261730    2558 flags.go:64] FLAG: --log-text-split-stream="false"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261733    2558 flags.go:64] FLAG: --logging-format="text"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261736    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261739    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261742    2558 flags.go:64] FLAG: --manifest-url=""
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261745    2558 flags.go:64] FLAG: --manifest-url-header=""
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261749    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261757    2558 flags.go:64] FLAG: --max-open-files="1000000"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261762    2558 flags.go:64] FLAG: --max-pods="110"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261765    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261768    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261772    2558 flags.go:64] FLAG: --memory-manager-policy="None"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261775    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261778    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261781    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261799    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261807    2558 flags.go:64] FLAG: --node-status-max-images="50"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261810    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261813    2558 flags.go:64] FLAG: --oom-score-adj="-999"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261816    2558 flags.go:64] FLAG: --pod-cidr=""
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261819    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 21 17:42:40.266073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261824    2558 flags.go:64] FLAG: --pod-manifest-path=""
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261827    2558 flags.go:64] FLAG: --pod-max-pids="-1"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261830    2558 flags.go:64] FLAG: --pods-per-core="0"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261834    2558 flags.go:64] FLAG: --port="10250"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261837    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261840    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fc00cf36349a5f38"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261843    2558 flags.go:64] FLAG: --qos-reserved=""
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261846    2558 flags.go:64] FLAG: --read-only-port="10255"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261849    2558 flags.go:64] FLAG: --register-node="true"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261852    2558 flags.go:64] FLAG: --register-schedulable="true"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261855    2558 flags.go:64] FLAG: --register-with-taints=""
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261859    2558 flags.go:64] FLAG: --registry-burst="10"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261862    2558 flags.go:64] FLAG: --registry-qps="5"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261864    2558 flags.go:64] FLAG: --reserved-cpus=""
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261867    2558 flags.go:64] FLAG: --reserved-memory=""
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261871    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261874    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261877    2558 flags.go:64] FLAG: --rotate-certificates="false"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261880    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261889    2558 flags.go:64] FLAG: --runonce="false"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261892    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261896    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261899    2558 flags.go:64] FLAG: --seccomp-default="false"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261902    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261905    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261908    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 21 17:42:40.266633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261911    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261914    2558 flags.go:64] FLAG: --storage-driver-password="root"
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261916    2558 flags.go:64] FLAG: --storage-driver-secure="false"
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261919    2558 flags.go:64] FLAG: --storage-driver-table="stats"
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261922    2558 flags.go:64] FLAG: --storage-driver-user="root"
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261925    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261928    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261931    2558 flags.go:64] FLAG: --system-cgroups=""
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261934    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261940    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261943    2558 flags.go:64] FLAG: --tls-cert-file=""
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261946    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261955    2558 flags.go:64] FLAG: --tls-min-version=""
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261959    2558 flags.go:64] FLAG: --tls-private-key-file=""
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261962    2558 flags.go:64] FLAG: --topology-manager-policy="none"
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261964    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261967    2558 flags.go:64] FLAG: --topology-manager-scope="container"
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261970    2558 flags.go:64] FLAG: --v="2"
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261975    2558 flags.go:64] FLAG: --version="false"
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261979    2558 flags.go:64] FLAG: --vmodule=""
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261983    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.261986    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262075    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262078    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 21 17:42:40.267295 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262082    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262085    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262094    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262097    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262100    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262103    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262106    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262108    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262111    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262114    2558 feature_gate.go:328] unrecognized feature gate: Example2
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262116    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262119    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262122    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262124    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262127    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262129    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262132    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262136    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262139    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262142    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 21 17:42:40.267928 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262145    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262147    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262150    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262152    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262155    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262157    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262160    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262163    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262165    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262168    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262171    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262173    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262176    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262179    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262181    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262190    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262195    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262197    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262200    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 21 17:42:40.268583 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262203    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262206    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262210    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262213    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262216    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262219    2558 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262222    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262225    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262227    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262230    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262234    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262237    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262240    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262243    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262245    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262248    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262258    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262261    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262264    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 21 17:42:40.269415 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262266    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262269    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262272    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262275    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262277    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262280    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262283    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262285    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262288    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262290    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262301    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262303    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262306    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262308    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262311    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262314    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262316    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262319    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262321    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262324    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 21 17:42:40.270243 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262326    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 21 17:42:40.271136 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262329    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 21 17:42:40.271136 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262332    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 21 17:42:40.271136 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262335    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 21 17:42:40.271136 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262338    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 21 17:42:40.271136 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.262341    2558 feature_gate.go:328] unrecognized feature gate: Example
May 21 17:42:40.271136 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.263087    2558 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 21 17:42:40.271136 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.270355    2558 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 21 17:42:40.271136 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.270376    2558 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 21 17:42:40.271136 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270456    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 21 17:42:40.271136 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270464    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 21 17:42:40.271136 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270469    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 21 17:42:40.271136 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270474    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 21 17:42:40.271136 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270479    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 21 17:42:40.271136 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270485    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 21 17:42:40.271136 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270489    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 21 17:42:40.271136 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270493    2558 feature_gate.go:328] unrecognized feature gate: Example2
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270497    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270502    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270506    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270510    2558 feature_gate.go:328] unrecognized feature gate: Example
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270514    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270518    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270522    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270529    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270536    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270541    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270546    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270551    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270555    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270559    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270563    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270568    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270572    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270576    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270581    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 21 17:42:40.271863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270585    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270589    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270594    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270598    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270604    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270609    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270613    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270618    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270622    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270626    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270631    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270635    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270639    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270644    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270648    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270653    2558 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270658    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270663    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270667    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 21 17:42:40.272398 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270671    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270675    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270680    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270685    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270689    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270693    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270698    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270702    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270706    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270711    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270715    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270720    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270724    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270728    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270733    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270738    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270742    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270749    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270757    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 21 17:42:40.273094 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270763    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270767    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270772    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270776    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270798    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270803    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270807    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270812    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270816    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270821    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270825    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270829    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270833    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270837    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270841    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270845    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270849    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270853    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270858    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270862    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 21 17:42:40.273558 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.270867    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 21 17:42:40.274102 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.270875    2558 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 21 17:42:40.274102 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271050    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 21 17:42:40.274102 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271059    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 21 17:42:40.274102 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271064    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 21 17:42:40.274102 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271069    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 21 17:42:40.274102 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271076    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 21 17:42:40.274102 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271082    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 21 17:42:40.274102 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271087    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 21 17:42:40.274102 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271092    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 21 17:42:40.274102 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271097    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 21 17:42:40.274102 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271102    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 21 17:42:40.274102 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271107    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 21 17:42:40.274102 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271112    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 21 17:42:40.274102 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271117    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 21 17:42:40.274102 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271121    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271125    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271129    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271133    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271137    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271141    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271145    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271149    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271153    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271157    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271162    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271166    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271170    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271175    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271179    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271184    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271188    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271192    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271196    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271201    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 21 17:42:40.274493 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271206    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271210    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271215    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271219    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271223    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271228    2558 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271232    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271236    2558 feature_gate.go:328] unrecognized feature gate: Example
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271240    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271244    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271250    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271255    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271259    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271263    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271267    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271272    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271276    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271280    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271285    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271289    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 21 17:42:40.275062 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271293    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271298    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271302    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271306    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271310    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271314    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271319    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271323    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271327    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271331    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271335    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271340    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271345    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271349    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271354    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271358    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271362    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271366    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271370    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 21 17:42:40.275913 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271374    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
May 21 17:42:40.276411 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271379    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 21 17:42:40.276411 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271383    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 21 17:42:40.276411 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271387    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 21 17:42:40.276411 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271393    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 21 17:42:40.276411 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271398    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 21 17:42:40.276411 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271402    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 21 17:42:40.276411 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271406    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 21 17:42:40.276411 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271413    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 21 17:42:40.276411 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271418    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 21 17:42:40.276411 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271422    2558 feature_gate.go:328] unrecognized feature gate: Example2
May 21 17:42:40.276411 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271427    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 21 17:42:40.276411 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271431    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 21 17:42:40.276411 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:40.271436    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 21 17:42:40.276411 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.271444    2558 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 21 17:42:40.276411 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.271604    2558 server.go:962] "Client rotation is on, will bootstrap in background"
May 21 17:42:40.277179 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.277164    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 21 17:42:40.278610 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.278597    2558 server.go:1019] "Starting client certificate rotation"
May 21 17:42:40.278713 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.278696    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 21 17:42:40.278756 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.278734    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 21 17:42:40.306819 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.306773    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 21 17:42:40.308168 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.308147    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 21 17:42:40.323296 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.323276    2558 log.go:25] "Validated CRI v1 runtime API"
May 21 17:42:40.328665 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.328645    2558 log.go:25] "Validated CRI v1 image API"
May 21 17:42:40.332181 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.332155    2558 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 21 17:42:40.335625 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.335605    2558 fs.go:135] Filesystem UUIDs: map[5c36ceb5-3198-44be-8daa-f9d93ded3620:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7b1b6221-4f04-478a-9cf4-1578d0286759:/dev/nvme0n1p4]
May 21 17:42:40.335684 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.335626    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 21 17:42:40.341363 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.341341    2558 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 21 17:42:40.341463 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.340432    2558 manager.go:217] Machine: {Timestamp:2026-05-21 17:42:40.339931288 +0000 UTC m=+0.450324263 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100087 MemoryCapacity:33164505088 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22be8d3da62414f252263ada0fd7d1 SystemUUID:ec22be8d-3da6-2414-f252-263ada0fd7d1 BootID:dfb8b76e-7a02-4f52-9679-46f5c211e05f Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582254592 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3e:e2:39:b9:77 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3e:e2:39:b9:77 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:2e:ce:4d:d6:54 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164505088 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 21 17:42:40.342243 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.342233    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 21 17:42:40.342341 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.342328    2558 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 21 17:42:40.345823 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.345799    2558 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 21 17:42:40.345969 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.345824    2558 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-125.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 21 17:42:40.346017 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.345976    2558 topology_manager.go:138] "Creating topology manager with none policy"
May 21 17:42:40.346017 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.345985    2558 container_manager_linux.go:306] "Creating device plugin manager"
May 21 17:42:40.346017 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.345993    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 21 17:42:40.346017 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.346009    2558 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 21 17:42:40.347570 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.347560    2558 state_mem.go:36] "Initialized new in-memory state store"
May 21 17:42:40.347680 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.347671    2558 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 21 17:42:40.350090 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.350081    2558 kubelet.go:491] "Attempting to sync node with API server"
May 21 17:42:40.350148 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.350093    2558 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 21 17:42:40.350653 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.350643    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 21 17:42:40.350687 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.350658    2558 kubelet.go:397] "Adding apiserver pod source"
May 21 17:42:40.350687 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.350668    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 21 17:42:40.352336 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.352312    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 21 17:42:40.352411 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.352349    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 21 17:42:40.356016 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.356000    2558 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 21 17:42:40.357945 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.357932    2558 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 21 17:42:40.358921 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.358908    2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 21 17:42:40.358962 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.358928    2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 21 17:42:40.358962 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.358937    2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 21 17:42:40.358962 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.358943    2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 21 17:42:40.358962 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.358952    2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 21 17:42:40.358962 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.358957    2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 21 17:42:40.358962 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.358963    2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 21 17:42:40.359123 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.358969    2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 21 17:42:40.359123 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.358975    2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 21 17:42:40.359123 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.358981    2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 21 17:42:40.359123 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.358997    2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 21 17:42:40.359241 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.359233    2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 21 17:42:40.360088 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.360080    2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 21 17:42:40.360088 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.360089    2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 21 17:42:40.362769 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.362580    2558 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 21 17:42:40.363192 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.363170    2558 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-125.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 21 17:42:40.363682 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.363669    2558 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 21 17:42:40.363738 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.363722    2558 server.go:1295] "Started kubelet"
May 21 17:42:40.363809 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.363762    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 21 17:42:40.363929 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.363864    2558 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 21 17:42:40.363980 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.363954    2558 server_v1.go:47] "podresources" method="list" useActivePods=true
May 21 17:42:40.364524 ip-10-0-135-125 systemd[1]: Started Kubernetes Kubelet.
May 21 17:42:40.366141 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.366120    2558 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-125.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 21 17:42:40.366248 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.366234    2558 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 21 17:42:40.367398 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.367374    2558 server.go:317] "Adding debug handlers to kubelet server"
May 21 17:42:40.373981 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.373074    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-125.ec2.internal.18b1a58ec301fbbd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-125.ec2.internal,UID:ip-10-0-135-125.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-125.ec2.internal,},FirstTimestamp:2026-05-21 17:42:40.363682749 +0000 UTC m=+0.474075720,LastTimestamp:2026-05-21 17:42:40.363682749 +0000 UTC m=+0.474075720,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-125.ec2.internal,}"
May 21 17:42:40.374139 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.374003    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 21 17:42:40.374139 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.374003    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 21 17:42:40.374250 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.374168    2558 volume_manager.go:295] "The desired_state_of_world populator starts"
May 21 17:42:40.374250 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.374183    2558 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 21 17:42:40.374380 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.374310    2558 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 21 17:42:40.374380 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.374375    2558 reconstruct.go:97] "Volume reconstruction finished"
May 21 17:42:40.374490 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.374384    2558 reconciler.go:26] "Reconciler: start to sync state"
May 21 17:42:40.374727 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.374696    2558 factory.go:55] Registering systemd factory
May 21 17:42:40.374727 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.374714    2558 factory.go:223] Registration of the systemd container factory successfully
May 21 17:42:40.374881 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.374832    2558 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 21 17:42:40.375069 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.375050    2558 factory.go:153] Registering CRI-O factory
May 21 17:42:40.375069 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.375070    2558 factory.go:223] Registration of the crio container factory successfully
May 21 17:42:40.375223 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.375118    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 21 17:42:40.375223 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.375135    2558 factory.go:103] Registering Raw factory
May 21 17:42:40.375223 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.375154    2558 manager.go:1196] Started watching for new ooms in manager
May 21 17:42:40.375579 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.375563    2558 manager.go:319] Starting recovery of all containers
May 21 17:42:40.376535 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.375053    2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
May 21 17:42:40.379602 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.379578    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b66qt"
May 21 17:42:40.382379 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.382350    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-125.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
May 21 17:42:40.382474 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.382442    2558 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
May 21 17:42:40.385925 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.385910    2558 manager.go:324] Recovery completed
May 21 17:42:40.387443 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.387422    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b66qt"
May 21 17:42:40.390592 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.390493    2558 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 21 17:42:40.392609 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.392592    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientMemory"
May 21 17:42:40.392700 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.392620    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasNoDiskPressure"
May 21 17:42:40.392700 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.392635    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientPID"
May 21 17:42:40.393119 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.393107    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 21 17:42:40.393119 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.393118    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 21 17:42:40.393191 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.393132    2558 state_mem.go:36] "Initialized new in-memory state store"
May 21 17:42:40.394778 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.394716    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-125.ec2.internal.18b1a58ec4bb5a29  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-125.ec2.internal,UID:ip-10-0-135-125.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-125.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-125.ec2.internal,},FirstTimestamp:2026-05-21 17:42:40.392608297 +0000 UTC m=+0.503001268,LastTimestamp:2026-05-21 17:42:40.392608297 +0000 UTC m=+0.503001268,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-125.ec2.internal,}"
May 21 17:42:40.396016 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.396004    2558 policy_none.go:49] "None policy: Start"
May 21 17:42:40.396059 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.396021    2558 memory_manager.go:186] "Starting memorymanager" policy="None"
May 21 17:42:40.396059 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.396032    2558 state_mem.go:35] "Initializing new in-memory state store"
May 21 17:42:40.434378 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.434363    2558 manager.go:341] "Starting Device Plugin manager"
May 21 17:42:40.442532 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.434400    2558 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 21 17:42:40.442532 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.434410    2558 server.go:85] "Starting device plugin registration server"
May 21 17:42:40.442532 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.434635    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
May 21 17:42:40.442532 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.434646    2558 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 21 17:42:40.442532 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.434736    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 21 17:42:40.442532 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.434834    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 21 17:42:40.442532 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.434844    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 21 17:42:40.442532 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.435327    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 21 17:42:40.442532 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.435363    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-125.ec2.internal\" not found"
May 21 17:42:40.501043 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.501007    2558 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 21 17:42:40.502170 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.502155    2558 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 21 17:42:40.502170 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.502173    2558 status_manager.go:230] "Starting to sync pod status with apiserver"
May 21 17:42:40.502333 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.502190    2558 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 21 17:42:40.502333 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.502197    2558 kubelet.go:2451] "Starting kubelet main sync loop"
May 21 17:42:40.502333 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.502233    2558 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 21 17:42:40.504712 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.504659    2558 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 21 17:42:40.535478 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.535461    2558 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 21 17:42:40.537426 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.537409    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientMemory"
May 21 17:42:40.537521 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.537444    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasNoDiskPressure"
May 21 17:42:40.537521 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.537458    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientPID"
May 21 17:42:40.537521 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.537490    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-125.ec2.internal"
May 21 17:42:40.543778 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.543763    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-125.ec2.internal"
May 21 17:42:40.543853 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.543799    2558 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-125.ec2.internal\": node \"ip-10-0-135-125.ec2.internal\" not found"
May 21 17:42:40.559621 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.559600    2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
May 21 17:42:40.602828 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.602805    2558 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal"]
May 21 17:42:40.602919 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.602879    2558 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 21 17:42:40.605091 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.605073    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientMemory"
May 21 17:42:40.605161 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.605103    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasNoDiskPressure"
May 21 17:42:40.605161 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.605112    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientPID"
May 21 17:42:40.606452 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.606439    2558 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 21 17:42:40.606957 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.606596    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
May 21 17:42:40.606957 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.606633    2558 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 21 17:42:40.607389 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.607374    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientMemory"
May 21 17:42:40.607473 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.607389    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientMemory"
May 21 17:42:40.607473 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.607402    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasNoDiskPressure"
May 21 17:42:40.607473 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.607412    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasNoDiskPressure"
May 21 17:42:40.607473 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.607427    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientPID"
May 21 17:42:40.607473 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.607413    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientPID"
May 21 17:42:40.608575 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.608556    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal"
May 21 17:42:40.608654 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.608581    2558 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 21 17:42:40.609218 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.609208    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientMemory"
May 21 17:42:40.609284 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.609230    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasNoDiskPressure"
May 21 17:42:40.609284 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.609238    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeHasSufficientPID"
May 21 17:42:40.624361 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.624340    2558 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-125.ec2.internal\" not found" node="ip-10-0-135-125.ec2.internal"
May 21 17:42:40.628113 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.628095    2558 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-125.ec2.internal\" not found" node="ip-10-0-135-125.ec2.internal"
May 21 17:42:40.659905 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.659881    2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
May 21 17:42:40.676338 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.676313    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/620cacdc789a7ea51780657f1a49d74a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal\" (UID: \"620cacdc789a7ea51780657f1a49d74a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
May 21 17:42:40.676415 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.676343    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/620cacdc789a7ea51780657f1a49d74a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal\" (UID: \"620cacdc789a7ea51780657f1a49d74a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
May 21 17:42:40.676415 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.676358    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/49c69236e53b609f3a6575a47767eea0-config\") pod \"kube-apiserver-proxy-ip-10-0-135-125.ec2.internal\" (UID: \"49c69236e53b609f3a6575a47767eea0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal"
May 21 17:42:40.760022 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.759932    2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
May 21 17:42:40.777349 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.777316    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/620cacdc789a7ea51780657f1a49d74a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal\" (UID: \"620cacdc789a7ea51780657f1a49d74a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
May 21 17:42:40.777457 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.777276    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/620cacdc789a7ea51780657f1a49d74a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal\" (UID: \"620cacdc789a7ea51780657f1a49d74a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
May 21 17:42:40.777457 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.777393    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/620cacdc789a7ea51780657f1a49d74a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal\" (UID: \"620cacdc789a7ea51780657f1a49d74a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
May 21 17:42:40.777457 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.777420    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/49c69236e53b609f3a6575a47767eea0-config\") pod \"kube-apiserver-proxy-ip-10-0-135-125.ec2.internal\" (UID: \"49c69236e53b609f3a6575a47767eea0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal"
May 21 17:42:40.777579 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.777486    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/49c69236e53b609f3a6575a47767eea0-config\") pod \"kube-apiserver-proxy-ip-10-0-135-125.ec2.internal\" (UID: \"49c69236e53b609f3a6575a47767eea0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal"
May 21 17:42:40.777579 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.777493    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/620cacdc789a7ea51780657f1a49d74a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal\" (UID: \"620cacdc789a7ea51780657f1a49d74a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
May 21 17:42:40.860715 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.860682    2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
May 21 17:42:40.927156 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.927127    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
May 21 17:42:40.930942 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:40.930923    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal"
May 21 17:42:40.961084 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:40.960914    2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
May 21 17:42:41.061477 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:41.061377    2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
May 21 17:42:41.161902 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:41.161860    2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
May 21 17:42:41.262457 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:41.262419    2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
May 21 17:42:41.278728 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:41.278701    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 21 17:42:41.278886 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:41.278871    2558 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 21 17:42:41.363407 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:41.363346    2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
May 21 17:42:41.374093 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:41.374069    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 21 17:42:41.383107 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:41.383087    2558 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 21 17:42:41.389030 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:41.389005    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-20 17:37:40 +0000 UTC" deadline="2028-01-29 15:26:32.335084358 +0000 UTC"
May 21 17:42:41.389109 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:41.389030    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14829h43m50.946058157s"
May 21 17:42:41.400879 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:41.400859    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-25h2n"
May 21 17:42:41.409994 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:41.409975    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-25h2n"
May 21 17:42:41.415134 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:41.415114    2558 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 21 17:42:41.464138 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:41.464104    2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
May 21 17:42:41.510331 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:41.510293    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49c69236e53b609f3a6575a47767eea0.slice/crio-942a85e3e333e1ab4918254e179c5ffe83341c1f0f5dc9b82d566f3a1a2a3cf3 WatchSource:0}: Error finding container 942a85e3e333e1ab4918254e179c5ffe83341c1f0f5dc9b82d566f3a1a2a3cf3: Status 404 returned error can't find the container with id 942a85e3e333e1ab4918254e179c5ffe83341c1f0f5dc9b82d566f3a1a2a3cf3
May 21 17:42:41.510617 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:41.510596    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod620cacdc789a7ea51780657f1a49d74a.slice/crio-8bf9d57b7c604a58f364dcc4ee8233e0419ea62cada991aaae44eda80dc249ec WatchSource:0}: Error finding container 8bf9d57b7c604a58f364dcc4ee8233e0419ea62cada991aaae44eda80dc249ec: Status 404 returned error can't find the container with id 8bf9d57b7c604a58f364dcc4ee8233e0419ea62cada991aaae44eda80dc249ec
May 21 17:42:41.514215 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:41.514201    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 21 17:42:41.564870 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:41.564837    2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-125.ec2.internal\" not found"
May 21 17:42:41.629939 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:41.629882    2558 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 21 17:42:41.674578 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:41.674554    2558 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal"
May 21 17:42:41.686280 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:41.686247    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 21 17:42:41.687550 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:41.687536    2558 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal"
May 21 17:42:41.694745 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:41.694733    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 21 17:42:41.807068 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:41.807037    2558 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 21 17:42:42.351566 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.351532    2558 apiserver.go:52] "Watching apiserver"
May 21 17:42:42.358227 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.358202    2558 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 21 17:42:42.359641 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.359609    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-brblc","kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp","openshift-cluster-node-tuning-operator/tuned-6wqpd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal","openshift-multus/multus-additional-cni-plugins-qtxx2","openshift-multus/network-metrics-daemon-sk2jf","openshift-network-operator/iptables-alerter-rmh5q","openshift-image-registry/node-ca-zcbrp","openshift-multus/multus-mkh56","openshift-network-diagnostics/network-check-target-brzgc","openshift-ovn-kubernetes/ovnkube-node-4ht7c"]
May 21 17:42:42.362629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.362607    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-brblc"
May 21 17:42:42.364894 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.364865    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.365658 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.365520    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 21 17:42:42.365658 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.365540    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 21 17:42:42.365658 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.365541    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nbnp9\""
May 21 17:42:42.366941 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.366554    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 21 17:42:42.366941 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.366812    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 21 17:42:42.367184 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.367116    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 21 17:42:42.367184 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.367171    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.367563 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.367398    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bw7sf\""
May 21 17:42:42.369061 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.369032    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 21 17:42:42.369154 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.369064    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 21 17:42:42.369154 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.369094    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nn4g4\""
May 21 17:42:42.369405 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.369333    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.371269 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.371249    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 21 17:42:42.371546 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.371457    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 21 17:42:42.371546 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.371462    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 21 17:42:42.373838 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.373644    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 21 17:42:42.373960 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.373940    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 21 17:42:42.374308 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.374288    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6js2m\""
May 21 17:42:42.377177 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.377158    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rmh5q"
May 21 17:42:42.377264 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.377202    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:42.377603 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:42.377560    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:42:42.379752 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.379730    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 21 17:42:42.379964 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.379946    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 21 17:42:42.380388 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.380197    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 21 17:42:42.380388 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.380236    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wrblw\""
May 21 17:42:42.382100 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.382078    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zcbrp"
May 21 17:42:42.382190 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.382084    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mkh56"
May 21 17:42:42.384495 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.384474    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 21 17:42:42.384586 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.384544    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 21 17:42:42.384641 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.384476    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 21 17:42:42.384690 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.384641    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c7lz2\""
May 21 17:42:42.384809 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.384772    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 21 17:42:42.384907 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.384884    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pw75r\""
May 21 17:42:42.386409 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386386    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/22e5cf05-8688-49c7-8390-94b6030b614c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.386544 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386423    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs\") pod \"network-metrics-daemon-sk2jf\" (UID: \"48a7b5cd-49ef-4177-83f1-80c264282bf9\") " pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:42.386544 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386453    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-socket-dir\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.386544 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386477    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-device-dir\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.386544 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386522    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22e5cf05-8688-49c7-8390-94b6030b614c-cnibin\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.386704 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386553    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2c0d8636-3bdd-479b-a341-b7a8094b9641-host-slash\") pod \"iptables-alerter-rmh5q\" (UID: \"2c0d8636-3bdd-479b-a341-b7a8094b9641\") " pod="openshift-network-operator/iptables-alerter-rmh5q"
May 21 17:42:42.386704 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386589    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-etc-selinux\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.386704 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386610    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-sysctl-conf\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.386704 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386625    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-var-lib-kubelet\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.386704 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386639    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bcffaae1-115b-493e-8df1-53ea90f7ec4e-tmp\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.386704 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386675    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/22e5cf05-8688-49c7-8390-94b6030b614c-os-release\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.386704 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386703    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/22e5cf05-8688-49c7-8390-94b6030b614c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.387009 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386719    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2c0d8636-3bdd-479b-a341-b7a8094b9641-iptables-alerter-script\") pod \"iptables-alerter-rmh5q\" (UID: \"2c0d8636-3bdd-479b-a341-b7a8094b9641\") " pod="openshift-network-operator/iptables-alerter-rmh5q"
May 21 17:42:42.387009 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386739    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b01317a-6e08-4b66-838d-59b9b97f25ef-agent-certs\") pod \"konnectivity-agent-brblc\" (UID: \"0b01317a-6e08-4b66-838d-59b9b97f25ef\") " pod="kube-system/konnectivity-agent-brblc"
May 21 17:42:42.387009 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386777    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b01317a-6e08-4b66-838d-59b9b97f25ef-konnectivity-ca\") pod \"konnectivity-agent-brblc\" (UID: \"0b01317a-6e08-4b66-838d-59b9b97f25ef\") " pod="kube-system/konnectivity-agent-brblc"
May 21 17:42:42.387009 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386824    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj6bn\" (UniqueName: \"kubernetes.io/projected/22e5cf05-8688-49c7-8390-94b6030b614c-kube-api-access-tj6bn\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.387009 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386853    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-lib-modules\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.387009 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386915    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22e5cf05-8688-49c7-8390-94b6030b614c-cni-binary-copy\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.387009 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.386981    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.387343 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.387045    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-sys-fs\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.387343 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.387080    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8xjb\" (UniqueName: \"kubernetes.io/projected/9c81a540-d2cd-41e1-bb7f-e474458ece54-kube-api-access-f8xjb\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.387343 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.387129    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-sysctl-d\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.387343 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.387195    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-systemd\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.387343 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.387219    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-sys\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.387343 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.387261    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22e5cf05-8688-49c7-8390-94b6030b614c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.387343 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.387324    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-registration-dir\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.387657 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.387349    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-modprobe-d\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.387657 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.387499    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:42:42.387657 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:42.387552    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:42:42.387657 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.387605    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.387964 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.387945    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-run\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.388085 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.388064    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45hbd\" (UniqueName: \"kubernetes.io/projected/48a7b5cd-49ef-4177-83f1-80c264282bf9-kube-api-access-45hbd\") pod \"network-metrics-daemon-sk2jf\" (UID: \"48a7b5cd-49ef-4177-83f1-80c264282bf9\") " pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:42.388181 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.388103    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgrrg\" (UniqueName: \"kubernetes.io/projected/2c0d8636-3bdd-479b-a341-b7a8094b9641-kube-api-access-tgrrg\") pod \"iptables-alerter-rmh5q\" (UID: \"2c0d8636-3bdd-479b-a341-b7a8094b9641\") " pod="openshift-network-operator/iptables-alerter-rmh5q"
May 21 17:42:42.388181 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.388131    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-kubernetes\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.388181 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.388170    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-sysconfig\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.388343 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.388195    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-tuned\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.388343 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.388237    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5hjq\" (UniqueName: \"kubernetes.io/projected/bcffaae1-115b-493e-8df1-53ea90f7ec4e-kube-api-access-w5hjq\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.388343 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.388263    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22e5cf05-8688-49c7-8390-94b6030b614c-system-cni-dir\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.388343 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.388287    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-host\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.389395 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.389374    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 21 17:42:42.389480 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.389420    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 21 17:42:42.389664 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.389647    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bcp45\""
May 21 17:42:42.389835 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.389817    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 21 17:42:42.389913 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.389875    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 21 17:42:42.389989 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.389970    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 21 17:42:42.390116 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.390099    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 21 17:42:42.410849 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.410825    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-20 17:37:41 +0000 UTC" deadline="2027-10-14 20:53:03.690297971 +0000 UTC"
May 21 17:42:42.410849 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.410846    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12267h10m21.279454088s"
May 21 17:42:42.475393 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.475363    2558 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 21 17:42:42.489360 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489333    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7rlw\" (UniqueName: \"kubernetes.io/projected/89c52e7d-700e-4c52-ad88-9a33c75f8f95-kube-api-access-j7rlw\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.489510 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489374    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tj6bn\" (UniqueName: \"kubernetes.io/projected/22e5cf05-8688-49c7-8390-94b6030b614c-kube-api-access-tj6bn\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.489510 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489405    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f8xjb\" (UniqueName: \"kubernetes.io/projected/9c81a540-d2cd-41e1-bb7f-e474458ece54-kube-api-access-f8xjb\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.489510 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489432    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-node-log\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.489663 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489541    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-sysctl-d\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.489663 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489570    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-systemd\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.489663 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489593    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-sys\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.489663 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489614    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-registration-dir\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.489663 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489638    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dswh8\" (UniqueName: \"kubernetes.io/projected/4f065c68-65f3-4014-8417-31619fc4d975-kube-api-access-dswh8\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.489663 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489661    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/511e916d-ee83-4b83-a9e4-811ff6092f9a-serviceca\") pod \"node-ca-zcbrp\" (UID: \"511e916d-ee83-4b83-a9e4-811ff6092f9a\") " pod="openshift-image-registry/node-ca-zcbrp"
May 21 17:42:42.489961 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489684    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-hostroot\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.489961 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489709    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/89c52e7d-700e-4c52-ad88-9a33c75f8f95-multus-daemon-config\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.489961 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489720    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-sysctl-d\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.489961 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489724    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-sys\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.489961 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489755    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-run\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.489961 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489768    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-systemd\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.489961 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489774    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-45hbd\" (UniqueName: \"kubernetes.io/projected/48a7b5cd-49ef-4177-83f1-80c264282bf9-kube-api-access-45hbd\") pod \"network-metrics-daemon-sk2jf\" (UID: \"48a7b5cd-49ef-4177-83f1-80c264282bf9\") " pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:42.489961 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489836    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-run-systemd\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.489961 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489850    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-registration-dir\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.489961 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489865    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-run-openvswitch\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.489961 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489894    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-run-k8s-cni-cncf-io\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.489961 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489922    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-run-netns\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.489961 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489948    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-var-lib-cni-multus\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.489961 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489849    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-run\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.490542 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.489986    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-etc-kubernetes\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.490542 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490021    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-var-lib-openvswitch\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.490542 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490048    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w5hjq\" (UniqueName: \"kubernetes.io/projected/bcffaae1-115b-493e-8df1-53ea90f7ec4e-kube-api-access-w5hjq\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.490542 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490066    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22e5cf05-8688-49c7-8390-94b6030b614c-system-cni-dir\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.490542 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490086    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-run-netns\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.490542 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490103    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4f065c68-65f3-4014-8417-31619fc4d975-ovnkube-config\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.490542 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490129    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-socket-dir\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.490542 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490153    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22e5cf05-8688-49c7-8390-94b6030b614c-system-cni-dir\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.490542 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490168    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-device-dir\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.490542 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490194    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-cni-netd\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.490542 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490203    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-device-dir\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.490542 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490218    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-system-cni-dir\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.490542 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490245    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-socket-dir\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.490542 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490246    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-etc-selinux\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.490542 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490302    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzhfd\" (UniqueName: \"kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd\") pod \"network-check-target-brzgc\" (UID: \"6db0adc0-a33d-4eca-988e-073a4bd19f80\") " pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:42:42.490542 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490307    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-etc-selinux\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490330    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-slash\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490352    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-cnibin\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490374    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-os-release\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490396    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-var-lib-kubelet\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490413    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/22e5cf05-8688-49c7-8390-94b6030b614c-os-release\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490429    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b01317a-6e08-4b66-838d-59b9b97f25ef-agent-certs\") pod \"konnectivity-agent-brblc\" (UID: \"0b01317a-6e08-4b66-838d-59b9b97f25ef\") " pod="kube-system/konnectivity-agent-brblc"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490445    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b01317a-6e08-4b66-838d-59b9b97f25ef-konnectivity-ca\") pod \"konnectivity-agent-brblc\" (UID: \"0b01317a-6e08-4b66-838d-59b9b97f25ef\") " pod="kube-system/konnectivity-agent-brblc"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490470    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-kubelet\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490504    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f065c68-65f3-4014-8417-31619fc4d975-ovn-node-metrics-cert\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490519    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/22e5cf05-8688-49c7-8390-94b6030b614c-os-release\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490529    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-lib-modules\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490556    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22e5cf05-8688-49c7-8390-94b6030b614c-cni-binary-copy\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490579    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-sys-fs\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490588    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-var-lib-kubelet\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490596    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4f065c68-65f3-4014-8417-31619fc4d975-env-overrides\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490611    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/511e916d-ee83-4b83-a9e4-811ff6092f9a-host\") pod \"node-ca-zcbrp\" (UID: \"511e916d-ee83-4b83-a9e4-811ff6092f9a\") " pod="openshift-image-registry/node-ca-zcbrp"
May 21 17:42:42.491282 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490643    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22e5cf05-8688-49c7-8390-94b6030b614c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490656    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-lib-modules\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490665    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-sys-fs\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490661    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-etc-openvswitch\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490767    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490892    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22e5cf05-8688-49c7-8390-94b6030b614c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490933    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-log-socket\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490968    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.490995    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw8fh\" (UniqueName: \"kubernetes.io/projected/511e916d-ee83-4b83-a9e4-811ff6092f9a-kube-api-access-qw8fh\") pod \"node-ca-zcbrp\" (UID: \"511e916d-ee83-4b83-a9e4-811ff6092f9a\") " pod="openshift-image-registry/node-ca-zcbrp"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491029    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-multus-conf-dir\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491062    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-run-multus-certs\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491102    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-modprobe-d\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491131    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tgrrg\" (UniqueName: \"kubernetes.io/projected/2c0d8636-3bdd-479b-a341-b7a8094b9641-kube-api-access-tgrrg\") pod \"iptables-alerter-rmh5q\" (UID: \"2c0d8636-3bdd-479b-a341-b7a8094b9641\") " pod="openshift-network-operator/iptables-alerter-rmh5q"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491189    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22e5cf05-8688-49c7-8390-94b6030b614c-cni-binary-copy\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491195    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0b01317a-6e08-4b66-838d-59b9b97f25ef-konnectivity-ca\") pod \"konnectivity-agent-brblc\" (UID: \"0b01317a-6e08-4b66-838d-59b9b97f25ef\") " pod="kube-system/konnectivity-agent-brblc"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491232    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-kubernetes\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491294    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4f065c68-65f3-4014-8417-31619fc4d975-ovnkube-script-lib\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.492046 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491320    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-modprobe-d\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491370    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-multus-cni-dir\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491416    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-sysconfig\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491418    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-kubernetes\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491455    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-tuned\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491496    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-multus-socket-dir-parent\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491523    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-host\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491521    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-sysconfig\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491549    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/22e5cf05-8688-49c7-8390-94b6030b614c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491580    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs\") pod \"network-metrics-daemon-sk2jf\" (UID: \"48a7b5cd-49ef-4177-83f1-80c264282bf9\") " pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491591    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-host\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491608    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-systemd-units\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:42.491679    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491717    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-cni-bin\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:42.491773    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs podName:48a7b5cd-49ef-4177-83f1-80c264282bf9 nodeName:}" failed. No retries permitted until 2026-05-21 17:42:42.991719246 +0000 UTC m=+3.102112221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs") pod "network-metrics-daemon-sk2jf" (UID: "48a7b5cd-49ef-4177-83f1-80c264282bf9") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491829    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-var-lib-kubelet\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491860    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22e5cf05-8688-49c7-8390-94b6030b614c-cnibin\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.492868 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491884    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2c0d8636-3bdd-479b-a341-b7a8094b9641-host-slash\") pod \"iptables-alerter-rmh5q\" (UID: \"2c0d8636-3bdd-479b-a341-b7a8094b9641\") " pod="openshift-network-operator/iptables-alerter-rmh5q"
May 21 17:42:42.493485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491912    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-run-ovn\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.493485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491920    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22e5cf05-8688-49c7-8390-94b6030b614c-cnibin\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.493485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491937    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-run-ovn-kubernetes\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.493485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491976    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-sysctl-conf\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.493485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.491978    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2c0d8636-3bdd-479b-a341-b7a8094b9641-host-slash\") pod \"iptables-alerter-rmh5q\" (UID: \"2c0d8636-3bdd-479b-a341-b7a8094b9641\") " pod="openshift-network-operator/iptables-alerter-rmh5q"
May 21 17:42:42.493485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.492002    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bcffaae1-115b-493e-8df1-53ea90f7ec4e-tmp\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.493485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.492029    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/22e5cf05-8688-49c7-8390-94b6030b614c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.493485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.492056    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2c0d8636-3bdd-479b-a341-b7a8094b9641-iptables-alerter-script\") pod \"iptables-alerter-rmh5q\" (UID: \"2c0d8636-3bdd-479b-a341-b7a8094b9641\") " pod="openshift-network-operator/iptables-alerter-rmh5q"
May 21 17:42:42.493485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.492083    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/22e5cf05-8688-49c7-8390-94b6030b614c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.493485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.492145    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.493485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.492174    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/89c52e7d-700e-4c52-ad88-9a33c75f8f95-cni-binary-copy\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.493485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.492178    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-sysctl-conf\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.493485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.492205    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-var-lib-cni-bin\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.493485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.492307    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c81a540-d2cd-41e1-bb7f-e474458ece54-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.493485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.492738    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2c0d8636-3bdd-479b-a341-b7a8094b9641-iptables-alerter-script\") pod \"iptables-alerter-rmh5q\" (UID: \"2c0d8636-3bdd-479b-a341-b7a8094b9641\") " pod="openshift-network-operator/iptables-alerter-rmh5q"
May 21 17:42:42.493485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.492964    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/22e5cf05-8688-49c7-8390-94b6030b614c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.494457 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.494258    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bcffaae1-115b-493e-8df1-53ea90f7ec4e-etc-tuned\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.494547 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.494516    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0b01317a-6e08-4b66-838d-59b9b97f25ef-agent-certs\") pod \"konnectivity-agent-brblc\" (UID: \"0b01317a-6e08-4b66-838d-59b9b97f25ef\") " pod="kube-system/konnectivity-agent-brblc"
May 21 17:42:42.494547 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.494522    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bcffaae1-115b-493e-8df1-53ea90f7ec4e-tmp\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.498762 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.498727    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj6bn\" (UniqueName: \"kubernetes.io/projected/22e5cf05-8688-49c7-8390-94b6030b614c-kube-api-access-tj6bn\") pod \"multus-additional-cni-plugins-qtxx2\" (UID: \"22e5cf05-8688-49c7-8390-94b6030b614c\") " pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.499020 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.498939    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5hjq\" (UniqueName: \"kubernetes.io/projected/bcffaae1-115b-493e-8df1-53ea90f7ec4e-kube-api-access-w5hjq\") pod \"tuned-6wqpd\" (UID: \"bcffaae1-115b-493e-8df1-53ea90f7ec4e\") " pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.499412 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.499389    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8xjb\" (UniqueName: \"kubernetes.io/projected/9c81a540-d2cd-41e1-bb7f-e474458ece54-kube-api-access-f8xjb\") pod \"aws-ebs-csi-driver-node-t2bnp\" (UID: \"9c81a540-d2cd-41e1-bb7f-e474458ece54\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.499498 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.499465    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45hbd\" (UniqueName: \"kubernetes.io/projected/48a7b5cd-49ef-4177-83f1-80c264282bf9-kube-api-access-45hbd\") pod \"network-metrics-daemon-sk2jf\" (UID: \"48a7b5cd-49ef-4177-83f1-80c264282bf9\") " pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:42.500188 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.500170    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgrrg\" (UniqueName: \"kubernetes.io/projected/2c0d8636-3bdd-479b-a341-b7a8094b9641-kube-api-access-tgrrg\") pod \"iptables-alerter-rmh5q\" (UID: \"2c0d8636-3bdd-479b-a341-b7a8094b9641\") " pod="openshift-network-operator/iptables-alerter-rmh5q"
May 21 17:42:42.506848 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.506796    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal" event={"ID":"620cacdc789a7ea51780657f1a49d74a","Type":"ContainerStarted","Data":"8bf9d57b7c604a58f364dcc4ee8233e0419ea62cada991aaae44eda80dc249ec"}
May 21 17:42:42.507763 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.507741    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal" event={"ID":"49c69236e53b609f3a6575a47767eea0","Type":"ContainerStarted","Data":"942a85e3e333e1ab4918254e179c5ffe83341c1f0f5dc9b82d566f3a1a2a3cf3"}
May 21 17:42:42.568926 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.568900    2558 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 21 17:42:42.592959 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.592920    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-multus-socket-dir-parent\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.593113 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.592968    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-systemd-units\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.593113 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.592993    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-cni-bin\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.593113 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593018    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-var-lib-kubelet\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.593113 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593032    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-multus-socket-dir-parent\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.593113 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593043    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-run-ovn\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.593113 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593082    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-run-ovn-kubernetes\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.593113 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593086    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-var-lib-kubelet\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.593113 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593087    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-systemd-units\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.593113 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593102    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-cni-bin\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593124    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-run-ovn-kubernetes\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593148    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-run-ovn\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593168    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/89c52e7d-700e-4c52-ad88-9a33c75f8f95-cni-binary-copy\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593194    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-var-lib-cni-bin\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593218    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j7rlw\" (UniqueName: \"kubernetes.io/projected/89c52e7d-700e-4c52-ad88-9a33c75f8f95-kube-api-access-j7rlw\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593241    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-node-log\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593267    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dswh8\" (UniqueName: \"kubernetes.io/projected/4f065c68-65f3-4014-8417-31619fc4d975-kube-api-access-dswh8\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593290    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/511e916d-ee83-4b83-a9e4-811ff6092f9a-serviceca\") pod \"node-ca-zcbrp\" (UID: \"511e916d-ee83-4b83-a9e4-811ff6092f9a\") " pod="openshift-image-registry/node-ca-zcbrp"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593304    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-var-lib-cni-bin\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593321    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-node-log\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593311    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-hostroot\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593347    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-hostroot\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593363    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/89c52e7d-700e-4c52-ad88-9a33c75f8f95-multus-daemon-config\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593406    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-run-systemd\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593431    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-run-openvswitch\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593471    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-run-k8s-cni-cncf-io\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593497    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-run-netns\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.593515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593520    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-run-systemd\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593539    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-var-lib-cni-multus\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593562    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-etc-kubernetes\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593571    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-run-k8s-cni-cncf-io\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593587    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-var-lib-openvswitch\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593615    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-run-openvswitch\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593614    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-run-netns\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593638    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-var-lib-cni-multus\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593648    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4f065c68-65f3-4014-8417-31619fc4d975-ovnkube-config\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593670    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-var-lib-openvswitch\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593688    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-cni-netd\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593705    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-etc-kubernetes\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593647    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-run-netns\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593705    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-run-netns\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593716    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-system-cni-dir\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593742    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/89c52e7d-700e-4c52-ad88-9a33c75f8f95-cni-binary-copy\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593749    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tzhfd\" (UniqueName: \"kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd\") pod \"network-check-target-brzgc\" (UID: \"6db0adc0-a33d-4eca-988e-073a4bd19f80\") " pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593741    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-cni-netd\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.594324 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593800    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-slash\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593801    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/511e916d-ee83-4b83-a9e4-811ff6092f9a-serviceca\") pod \"node-ca-zcbrp\" (UID: \"511e916d-ee83-4b83-a9e4-811ff6092f9a\") " pod="openshift-image-registry/node-ca-zcbrp"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593825    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-cnibin\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593829    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-system-cni-dir\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593850    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-slash\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593855    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-os-release\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593878    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-cnibin\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593895    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-kubelet\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593905    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-os-release\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593894    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/89c52e7d-700e-4c52-ad88-9a33c75f8f95-multus-daemon-config\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593923    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f065c68-65f3-4014-8417-31619fc4d975-ovn-node-metrics-cert\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593948    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-kubelet\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593954    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4f065c68-65f3-4014-8417-31619fc4d975-env-overrides\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.593977    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/511e916d-ee83-4b83-a9e4-811ff6092f9a-host\") pod \"node-ca-zcbrp\" (UID: \"511e916d-ee83-4b83-a9e4-811ff6092f9a\") " pod="openshift-image-registry/node-ca-zcbrp"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594010    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-etc-openvswitch\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594035    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-log-socket\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594061    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594096    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qw8fh\" (UniqueName: \"kubernetes.io/projected/511e916d-ee83-4b83-a9e4-811ff6092f9a-kube-api-access-qw8fh\") pod \"node-ca-zcbrp\" (UID: \"511e916d-ee83-4b83-a9e4-811ff6092f9a\") " pod="openshift-image-registry/node-ca-zcbrp"
May 21 17:42:42.595018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594101    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-etc-openvswitch\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.595629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594125    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-multus-conf-dir\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.595629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594167    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-multus-conf-dir\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.595629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594166    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-run-multus-certs\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.595629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594196    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-host-run-multus-certs\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.595629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594207    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4f065c68-65f3-4014-8417-31619fc4d975-ovnkube-script-lib\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.595629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594209    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4f065c68-65f3-4014-8417-31619fc4d975-ovnkube-config\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.595629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594255    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/511e916d-ee83-4b83-a9e4-811ff6092f9a-host\") pod \"node-ca-zcbrp\" (UID: \"511e916d-ee83-4b83-a9e4-811ff6092f9a\") " pod="openshift-image-registry/node-ca-zcbrp"
May 21 17:42:42.595629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594256    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.595629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594336    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4f065c68-65f3-4014-8417-31619fc4d975-log-socket\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.595629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594366    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-multus-cni-dir\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.595629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594450    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89c52e7d-700e-4c52-ad88-9a33c75f8f95-multus-cni-dir\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.595629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594688    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4f065c68-65f3-4014-8417-31619fc4d975-ovnkube-script-lib\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.595629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.594901    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4f065c68-65f3-4014-8417-31619fc4d975-env-overrides\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.596556 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.596536    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f065c68-65f3-4014-8417-31619fc4d975-ovn-node-metrics-cert\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.599066 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:42.599044    2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 21 17:42:42.599192 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:42.599071    2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 21 17:42:42.599192 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:42.599088    2558 projected.go:194] Error preparing data for projected volume kube-api-access-tzhfd for pod openshift-network-diagnostics/network-check-target-brzgc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 17:42:42.599192 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:42.599150    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd podName:6db0adc0-a33d-4eca-988e-073a4bd19f80 nodeName:}" failed. No retries permitted until 2026-05-21 17:42:43.09913185 +0000 UTC m=+3.209524811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tzhfd" (UniqueName: "kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd") pod "network-check-target-brzgc" (UID: "6db0adc0-a33d-4eca-988e-073a4bd19f80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 17:42:42.601285 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.601254    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dswh8\" (UniqueName: \"kubernetes.io/projected/4f065c68-65f3-4014-8417-31619fc4d975-kube-api-access-dswh8\") pod \"ovnkube-node-4ht7c\" (UID: \"4f065c68-65f3-4014-8417-31619fc4d975\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.603985 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.601856    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7rlw\" (UniqueName: \"kubernetes.io/projected/89c52e7d-700e-4c52-ad88-9a33c75f8f95-kube-api-access-j7rlw\") pod \"multus-mkh56\" (UID: \"89c52e7d-700e-4c52-ad88-9a33c75f8f95\") " pod="openshift-multus/multus-mkh56"
May 21 17:42:42.603985 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.602031    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw8fh\" (UniqueName: \"kubernetes.io/projected/511e916d-ee83-4b83-a9e4-811ff6092f9a-kube-api-access-qw8fh\") pod \"node-ca-zcbrp\" (UID: \"511e916d-ee83-4b83-a9e4-811ff6092f9a\") " pod="openshift-image-registry/node-ca-zcbrp"
May 21 17:42:42.680311 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.680271    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-brblc"
May 21 17:42:42.689076 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.689052    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp"
May 21 17:42:42.697817 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.697799    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6wqpd"
May 21 17:42:42.703408 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.703385    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qtxx2"
May 21 17:42:42.710871 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.710853    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rmh5q"
May 21 17:42:42.716369 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.716350    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zcbrp"
May 21 17:42:42.723888 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.723870    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mkh56"
May 21 17:42:42.729436 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.729421    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:42:42.996147 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:42.996057    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs\") pod \"network-metrics-daemon-sk2jf\" (UID: \"48a7b5cd-49ef-4177-83f1-80c264282bf9\") " pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:42.996285 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:42.996186    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 17:42:42.996285 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:42.996243    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs podName:48a7b5cd-49ef-4177-83f1-80c264282bf9 nodeName:}" failed. No retries permitted until 2026-05-21 17:42:43.996229391 +0000 UTC m=+4.106622348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs") pod "network-metrics-daemon-sk2jf" (UID: "48a7b5cd-49ef-4177-83f1-80c264282bf9") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 17:42:43.198458 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:43.198417    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tzhfd\" (UniqueName: \"kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd\") pod \"network-check-target-brzgc\" (UID: \"6db0adc0-a33d-4eca-988e-073a4bd19f80\") " pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:42:43.198605 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:43.198571    2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 21 17:42:43.198605 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:43.198589    2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 21 17:42:43.198605 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:43.198598    2558 projected.go:194] Error preparing data for projected volume kube-api-access-tzhfd for pod openshift-network-diagnostics/network-check-target-brzgc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 17:42:43.198715 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:43.198648    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd podName:6db0adc0-a33d-4eca-988e-073a4bd19f80 nodeName:}" failed. No retries permitted until 2026-05-21 17:42:44.198634159 +0000 UTC m=+4.309027122 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzhfd" (UniqueName: "kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd") pod "network-check-target-brzgc" (UID: "6db0adc0-a33d-4eca-988e-073a4bd19f80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 17:42:43.252135 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:43.252110    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22e5cf05_8688_49c7_8390_94b6030b614c.slice/crio-f9d820200fe851e661dcc1a15cb6704b2795c84cf21ebb950096b9e0a915b10e WatchSource:0}: Error finding container f9d820200fe851e661dcc1a15cb6704b2795c84cf21ebb950096b9e0a915b10e: Status 404 returned error can't find the container with id f9d820200fe851e661dcc1a15cb6704b2795c84cf21ebb950096b9e0a915b10e
May 21 17:42:43.253023 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:43.252998    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f065c68_65f3_4014_8417_31619fc4d975.slice/crio-98f2b92be3370d1a55fbe5be5d6781cd300169c5794353e970860539573ab214 WatchSource:0}: Error finding container 98f2b92be3370d1a55fbe5be5d6781cd300169c5794353e970860539573ab214: Status 404 returned error can't find the container with id 98f2b92be3370d1a55fbe5be5d6781cd300169c5794353e970860539573ab214
May 21 17:42:43.255910 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:43.255887    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcffaae1_115b_493e_8df1_53ea90f7ec4e.slice/crio-f51f87b6dd8db60a68abe67b7e2b52180a0a68aac668471a234f9c956b10c411 WatchSource:0}: Error finding container f51f87b6dd8db60a68abe67b7e2b52180a0a68aac668471a234f9c956b10c411: Status 404 returned error can't find the container with id f51f87b6dd8db60a68abe67b7e2b52180a0a68aac668471a234f9c956b10c411
May 21 17:42:43.259381 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:43.259164    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89c52e7d_700e_4c52_ad88_9a33c75f8f95.slice/crio-d6353712f37a300e66411028f7ec198af3952d845a7c7f9435d13687f0f9f153 WatchSource:0}: Error finding container d6353712f37a300e66411028f7ec198af3952d845a7c7f9435d13687f0f9f153: Status 404 returned error can't find the container with id d6353712f37a300e66411028f7ec198af3952d845a7c7f9435d13687f0f9f153
May 21 17:42:43.259863 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:43.259838    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b01317a_6e08_4b66_838d_59b9b97f25ef.slice/crio-ae3a3a9c896a032fd07ba6ab88bbfb0d2e056e87f5d9de199e40eab2c2f44910 WatchSource:0}: Error finding container ae3a3a9c896a032fd07ba6ab88bbfb0d2e056e87f5d9de199e40eab2c2f44910: Status 404 returned error can't find the container with id ae3a3a9c896a032fd07ba6ab88bbfb0d2e056e87f5d9de199e40eab2c2f44910
May 21 17:42:43.261617 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:43.261044    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511e916d_ee83_4b83_a9e4_811ff6092f9a.slice/crio-822cb616bee81a93183456e1cc58fec3419a0e237709b06a746c1b44cb385492 WatchSource:0}: Error finding container 822cb616bee81a93183456e1cc58fec3419a0e237709b06a746c1b44cb385492: Status 404 returned error can't find the container with id 822cb616bee81a93183456e1cc58fec3419a0e237709b06a746c1b44cb385492
May 21 17:42:43.262487 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:42:43.262218    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c81a540_d2cd_41e1_bb7f_e474458ece54.slice/crio-2532a0c7af733faddacb4eec1679ddc4ec1a03e88c1467b292e78b23cbf64ae8 WatchSource:0}: Error finding container 2532a0c7af733faddacb4eec1679ddc4ec1a03e88c1467b292e78b23cbf64ae8: Status 404 returned error can't find the container with id 2532a0c7af733faddacb4eec1679ddc4ec1a03e88c1467b292e78b23cbf64ae8
May 21 17:42:43.411457 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:43.411420    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-20 17:37:41 +0000 UTC" deadline="2028-02-17 06:39:17.790767598 +0000 UTC"
May 21 17:42:43.411457 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:43.411450    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15276h56m34.379319289s"
May 21 17:42:43.510334 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:43.510301    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-brblc" event={"ID":"0b01317a-6e08-4b66-838d-59b9b97f25ef","Type":"ContainerStarted","Data":"ae3a3a9c896a032fd07ba6ab88bbfb0d2e056e87f5d9de199e40eab2c2f44910"}
May 21 17:42:43.511267 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:43.511243    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c" event={"ID":"4f065c68-65f3-4014-8417-31619fc4d975","Type":"ContainerStarted","Data":"98f2b92be3370d1a55fbe5be5d6781cd300169c5794353e970860539573ab214"}
May 21 17:42:43.512188 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:43.512168    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtxx2" event={"ID":"22e5cf05-8688-49c7-8390-94b6030b614c","Type":"ContainerStarted","Data":"f9d820200fe851e661dcc1a15cb6704b2795c84cf21ebb950096b9e0a915b10e"}
May 21 17:42:43.513496 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:43.513476    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal" event={"ID":"49c69236e53b609f3a6575a47767eea0","Type":"ContainerStarted","Data":"643a6f36ff274fc5d0e0731f737a96c299a5550bbc7855ea98f9f8b7f696b777"}
May 21 17:42:43.514532 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:43.514502    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rmh5q" event={"ID":"2c0d8636-3bdd-479b-a341-b7a8094b9641","Type":"ContainerStarted","Data":"8b4864702f629899bcd332816f1140fee5c367424efd37a9779971cd5183f3b3"}
May 21 17:42:43.515523 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:43.515504    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp" event={"ID":"9c81a540-d2cd-41e1-bb7f-e474458ece54","Type":"ContainerStarted","Data":"2532a0c7af733faddacb4eec1679ddc4ec1a03e88c1467b292e78b23cbf64ae8"}
May 21 17:42:43.516416 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:43.516393    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zcbrp" event={"ID":"511e916d-ee83-4b83-a9e4-811ff6092f9a","Type":"ContainerStarted","Data":"822cb616bee81a93183456e1cc58fec3419a0e237709b06a746c1b44cb385492"}
May 21 17:42:43.517345 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:43.517327    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mkh56" event={"ID":"89c52e7d-700e-4c52-ad88-9a33c75f8f95","Type":"ContainerStarted","Data":"d6353712f37a300e66411028f7ec198af3952d845a7c7f9435d13687f0f9f153"}
May 21 17:42:43.518148 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:43.518130    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6wqpd" event={"ID":"bcffaae1-115b-493e-8df1-53ea90f7ec4e","Type":"ContainerStarted","Data":"f51f87b6dd8db60a68abe67b7e2b52180a0a68aac668471a234f9c956b10c411"}
May 21 17:42:44.003519 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:44.003339    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs\") pod \"network-metrics-daemon-sk2jf\" (UID: \"48a7b5cd-49ef-4177-83f1-80c264282bf9\") " pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:44.003681 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:44.003568    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 17:42:44.003681 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:44.003636    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs podName:48a7b5cd-49ef-4177-83f1-80c264282bf9 nodeName:}" failed. No retries permitted until 2026-05-21 17:42:46.00361668 +0000 UTC m=+6.114009652 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs") pod "network-metrics-daemon-sk2jf" (UID: "48a7b5cd-49ef-4177-83f1-80c264282bf9") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 17:42:44.205771 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:44.205066    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tzhfd\" (UniqueName: \"kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd\") pod \"network-check-target-brzgc\" (UID: \"6db0adc0-a33d-4eca-988e-073a4bd19f80\") " pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:42:44.205771 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:44.205269    2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 21 17:42:44.205771 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:44.205287    2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 21 17:42:44.205771 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:44.205300    2558 projected.go:194] Error preparing data for projected volume kube-api-access-tzhfd for pod openshift-network-diagnostics/network-check-target-brzgc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 17:42:44.205771 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:44.205381    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd podName:6db0adc0-a33d-4eca-988e-073a4bd19f80 nodeName:}" failed. No retries permitted until 2026-05-21 17:42:46.205337904 +0000 UTC m=+6.315730870 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzhfd" (UniqueName: "kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd") pod "network-check-target-brzgc" (UID: "6db0adc0-a33d-4eca-988e-073a4bd19f80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 17:42:44.503631 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:44.502916    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:44.503631 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:44.503058    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:42:44.503631 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:44.503437    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:42:44.503631 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:44.503524    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:42:44.531586 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:44.531548    2558 generic.go:358] "Generic (PLEG): container finished" podID="620cacdc789a7ea51780657f1a49d74a" containerID="42da0c04f486694f077a2d5b4598de4e49ae2e320b7eb9b5002458cf9b05a0f0" exitCode=0
May 21 17:42:44.532519 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:44.532492    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal" event={"ID":"620cacdc789a7ea51780657f1a49d74a","Type":"ContainerDied","Data":"42da0c04f486694f077a2d5b4598de4e49ae2e320b7eb9b5002458cf9b05a0f0"}
May 21 17:42:44.545561 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:44.545055    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-125.ec2.internal" podStartSLOduration=3.545038296 podStartE2EDuration="3.545038296s" podCreationTimestamp="2026-05-21 17:42:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 17:42:43.549502766 +0000 UTC m=+3.659895746" watchObservedRunningTime="2026-05-21 17:42:44.545038296 +0000 UTC m=+4.655431277"
May 21 17:42:45.547022 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:45.546187    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal" event={"ID":"620cacdc789a7ea51780657f1a49d74a","Type":"ContainerStarted","Data":"0471ef883eb678cefc71bae8d3c4ca8f19a39ad40e3c7d98619ef9e5413a3a15"}
May 21 17:42:46.021147 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:46.021063    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs\") pod \"network-metrics-daemon-sk2jf\" (UID: \"48a7b5cd-49ef-4177-83f1-80c264282bf9\") " pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:46.021312 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:46.021203    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 17:42:46.021312 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:46.021271    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs podName:48a7b5cd-49ef-4177-83f1-80c264282bf9 nodeName:}" failed. No retries permitted until 2026-05-21 17:42:50.021254248 +0000 UTC m=+10.131647206 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs") pod "network-metrics-daemon-sk2jf" (UID: "48a7b5cd-49ef-4177-83f1-80c264282bf9") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 17:42:46.222509 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:46.222477    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tzhfd\" (UniqueName: \"kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd\") pod \"network-check-target-brzgc\" (UID: \"6db0adc0-a33d-4eca-988e-073a4bd19f80\") " pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:42:46.222674 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:46.222620    2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 21 17:42:46.222674 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:46.222639    2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 21 17:42:46.222674 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:46.222652    2558 projected.go:194] Error preparing data for projected volume kube-api-access-tzhfd for pod openshift-network-diagnostics/network-check-target-brzgc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 17:42:46.222856 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:46.222718    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd podName:6db0adc0-a33d-4eca-988e-073a4bd19f80 nodeName:}" failed. No retries permitted until 2026-05-21 17:42:50.222697868 +0000 UTC m=+10.333090845 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzhfd" (UniqueName: "kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd") pod "network-check-target-brzgc" (UID: "6db0adc0-a33d-4eca-988e-073a4bd19f80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 17:42:46.503413 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:46.503339    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:46.503557 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:46.503489    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:42:46.504042 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:46.503867    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:42:46.504042 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:46.503961    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:42:48.503629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:48.503281    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:42:48.503629 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:48.503404    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:42:48.503629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:48.503429    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:48.503629 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:48.503552    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:42:50.054448 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:50.054407    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs\") pod \"network-metrics-daemon-sk2jf\" (UID: \"48a7b5cd-49ef-4177-83f1-80c264282bf9\") " pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:50.054935 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:50.054561    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 17:42:50.054935 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:50.054647    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs podName:48a7b5cd-49ef-4177-83f1-80c264282bf9 nodeName:}" failed. No retries permitted until 2026-05-21 17:42:58.054625553 +0000 UTC m=+18.165018516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs") pod "network-metrics-daemon-sk2jf" (UID: "48a7b5cd-49ef-4177-83f1-80c264282bf9") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 17:42:50.256350 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:50.256280    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tzhfd\" (UniqueName: \"kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd\") pod \"network-check-target-brzgc\" (UID: \"6db0adc0-a33d-4eca-988e-073a4bd19f80\") " pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:42:50.256518 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:50.256466    2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 21 17:42:50.256518 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:50.256485    2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 21 17:42:50.256518 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:50.256497    2558 projected.go:194] Error preparing data for projected volume kube-api-access-tzhfd for pod openshift-network-diagnostics/network-check-target-brzgc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 17:42:50.256689 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:50.256574    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd podName:6db0adc0-a33d-4eca-988e-073a4bd19f80 nodeName:}" failed. No retries permitted until 2026-05-21 17:42:58.256545181 +0000 UTC m=+18.366938161 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzhfd" (UniqueName: "kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd") pod "network-check-target-brzgc" (UID: "6db0adc0-a33d-4eca-988e-073a4bd19f80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 17:42:50.504098 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:50.503368    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:50.504098 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:50.503522    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:42:50.504098 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:50.503931    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:42:50.504098 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:50.504015    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:42:52.503089 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:52.503019    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:42:52.503089 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:52.503050    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:52.503577 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:52.503163    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:42:52.503692 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:52.503654    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:42:54.502414 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:54.502382    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:42:54.502855 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:54.502389    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:54.502855 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:54.502505    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:42:54.502855 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:54.502610    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:42:56.503223 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:56.503138    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:42:56.503612 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:56.503139    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:56.503612 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:56.503257    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:42:56.503612 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:56.503388    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:42:58.111835 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:58.111767    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs\") pod \"network-metrics-daemon-sk2jf\" (UID: \"48a7b5cd-49ef-4177-83f1-80c264282bf9\") " pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:58.112295 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:58.111918    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 17:42:58.112295 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:58.111983    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs podName:48a7b5cd-49ef-4177-83f1-80c264282bf9 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:14.111965199 +0000 UTC m=+34.222358169 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs") pod "network-metrics-daemon-sk2jf" (UID: "48a7b5cd-49ef-4177-83f1-80c264282bf9") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 17:42:58.313451 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:58.313413    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tzhfd\" (UniqueName: \"kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd\") pod \"network-check-target-brzgc\" (UID: \"6db0adc0-a33d-4eca-988e-073a4bd19f80\") " pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:42:58.313630 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:58.313576    2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 21 17:42:58.313630 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:58.313592    2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 21 17:42:58.313630 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:58.313602    2558 projected.go:194] Error preparing data for projected volume kube-api-access-tzhfd for pod openshift-network-diagnostics/network-check-target-brzgc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 17:42:58.313763 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:58.313655    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd podName:6db0adc0-a33d-4eca-988e-073a4bd19f80 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:14.313639033 +0000 UTC m=+34.424032007 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzhfd" (UniqueName: "kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd") pod "network-check-target-brzgc" (UID: "6db0adc0-a33d-4eca-988e-073a4bd19f80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 17:42:58.502975 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:58.502887    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:42:58.502975 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:42:58.502938    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:42:58.503167 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:58.503020    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:42:58.503218 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:42:58.503181    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:43:00.033138 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.033086    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-125.ec2.internal" podStartSLOduration=19.03307215 podStartE2EDuration="19.03307215s" podCreationTimestamp="2026-05-21 17:42:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 17:42:45.559763299 +0000 UTC m=+5.670156279" watchObservedRunningTime="2026-05-21 17:43:00.03307215 +0000 UTC m=+20.143465131"
May 21 17:43:00.033691 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.033226    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-h9lc4"]
May 21 17:43:00.159274 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.159245    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h9lc4"
May 21 17:43:00.161900 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.161872    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 21 17:43:00.162049 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.161928    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 21 17:43:00.162049 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.161929    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jhvpx\""
May 21 17:43:00.228049 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.228017    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f95b9ee8-6015-4709-ac02-d7db0b48a238-tmp-dir\") pod \"node-resolver-h9lc4\" (UID: \"f95b9ee8-6015-4709-ac02-d7db0b48a238\") " pod="openshift-dns/node-resolver-h9lc4"
May 21 17:43:00.228213 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.228079    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f95b9ee8-6015-4709-ac02-d7db0b48a238-hosts-file\") pod \"node-resolver-h9lc4\" (UID: \"f95b9ee8-6015-4709-ac02-d7db0b48a238\") " pod="openshift-dns/node-resolver-h9lc4"
May 21 17:43:00.228213 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.228099    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmkqs\" (UniqueName: \"kubernetes.io/projected/f95b9ee8-6015-4709-ac02-d7db0b48a238-kube-api-access-wmkqs\") pod \"node-resolver-h9lc4\" (UID: \"f95b9ee8-6015-4709-ac02-d7db0b48a238\") " pod="openshift-dns/node-resolver-h9lc4"
May 21 17:43:00.329255 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.329182    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f95b9ee8-6015-4709-ac02-d7db0b48a238-hosts-file\") pod \"node-resolver-h9lc4\" (UID: \"f95b9ee8-6015-4709-ac02-d7db0b48a238\") " pod="openshift-dns/node-resolver-h9lc4"
May 21 17:43:00.329255 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.329223    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wmkqs\" (UniqueName: \"kubernetes.io/projected/f95b9ee8-6015-4709-ac02-d7db0b48a238-kube-api-access-wmkqs\") pod \"node-resolver-h9lc4\" (UID: \"f95b9ee8-6015-4709-ac02-d7db0b48a238\") " pod="openshift-dns/node-resolver-h9lc4"
May 21 17:43:00.329432 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.329274    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f95b9ee8-6015-4709-ac02-d7db0b48a238-tmp-dir\") pod \"node-resolver-h9lc4\" (UID: \"f95b9ee8-6015-4709-ac02-d7db0b48a238\") " pod="openshift-dns/node-resolver-h9lc4"
May 21 17:43:00.329432 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.329332    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f95b9ee8-6015-4709-ac02-d7db0b48a238-hosts-file\") pod \"node-resolver-h9lc4\" (UID: \"f95b9ee8-6015-4709-ac02-d7db0b48a238\") " pod="openshift-dns/node-resolver-h9lc4"
May 21 17:43:00.330503 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.330488    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f95b9ee8-6015-4709-ac02-d7db0b48a238-tmp-dir\") pod \"node-resolver-h9lc4\" (UID: \"f95b9ee8-6015-4709-ac02-d7db0b48a238\") " pod="openshift-dns/node-resolver-h9lc4"
May 21 17:43:00.338483 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.338364    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmkqs\" (UniqueName: \"kubernetes.io/projected/f95b9ee8-6015-4709-ac02-d7db0b48a238-kube-api-access-wmkqs\") pod \"node-resolver-h9lc4\" (UID: \"f95b9ee8-6015-4709-ac02-d7db0b48a238\") " pod="openshift-dns/node-resolver-h9lc4"
May 21 17:43:00.468441 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.468417    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h9lc4"
May 21 17:43:00.506068 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.505959    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:43:00.506137 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:00.506061    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:43:00.506682 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.506488    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:43:00.506682 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:00.506589    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:43:00.592628 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.588881    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6wqpd" event={"ID":"bcffaae1-115b-493e-8df1-53ea90f7ec4e","Type":"ContainerStarted","Data":"9e7c9cc1e613fb26461be58c8b0a16315be1bed8276b71b273de3ded4c0e936e"}
May 21 17:43:00.594458 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.594431    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h9lc4" event={"ID":"f95b9ee8-6015-4709-ac02-d7db0b48a238","Type":"ContainerStarted","Data":"b51f2e82481d59714fe5140789751aad71a20e0e0089fb13f1777f8d631b23bf"}
May 21 17:43:00.598533 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.598299    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-brblc" event={"ID":"0b01317a-6e08-4b66-838d-59b9b97f25ef","Type":"ContainerStarted","Data":"f6e5905cf47ac8acbdf7770f7cb5952b98c804b3e8f382ab3ece86bf2f6e3700"}
May 21 17:43:00.618401 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.618353    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6wqpd" podStartSLOduration=3.483763316 podStartE2EDuration="20.618335731s" podCreationTimestamp="2026-05-21 17:42:40 +0000 UTC" firstStartedPulling="2026-05-21 17:42:43.259899345 +0000 UTC m=+3.370292320" lastFinishedPulling="2026-05-21 17:43:00.394471778 +0000 UTC m=+20.504864735" observedRunningTime="2026-05-21 17:43:00.606553222 +0000 UTC m=+20.716946203" watchObservedRunningTime="2026-05-21 17:43:00.618335731 +0000 UTC m=+20.728728723"
May 21 17:43:00.875351 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.875137    2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-brblc"
May 21 17:43:00.875914 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.875898    2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-brblc"
May 21 17:43:00.887562 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:00.887526    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-brblc" podStartSLOduration=3.7556506020000002 podStartE2EDuration="20.887513427s" podCreationTimestamp="2026-05-21 17:42:40 +0000 UTC" firstStartedPulling="2026-05-21 17:42:43.262574922 +0000 UTC m=+3.372967890" lastFinishedPulling="2026-05-21 17:43:00.394437749 +0000 UTC m=+20.504830715" observedRunningTime="2026-05-21 17:43:00.618258575 +0000 UTC m=+20.728651555" watchObservedRunningTime="2026-05-21 17:43:00.887513427 +0000 UTC m=+20.997906406"
May 21 17:43:01.601152 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.601075    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mkh56" event={"ID":"89c52e7d-700e-4c52-ad88-9a33c75f8f95","Type":"ContainerStarted","Data":"1175370a50aec74bd833b4149433bd2cc517c7dc026833e0f50489506516eab5"}
May 21 17:43:01.602334 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.602312    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h9lc4" event={"ID":"f95b9ee8-6015-4709-ac02-d7db0b48a238","Type":"ContainerStarted","Data":"9f88b802299fed1f98b3f35077db0f1cfe94d85f969c9e7eeb9f06a21cd6cacd"}
May 21 17:43:01.604658 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.604637    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 17:43:01.604972 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.604951    2558 generic.go:358] "Generic (PLEG): container finished" podID="4f065c68-65f3-4014-8417-31619fc4d975" containerID="d3502caa6c9aadf3f1449c66aaf03a3989fb639ed6c6fb47bd63fe89b6281ac2" exitCode=1
May 21 17:43:01.605047 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.605011    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c" event={"ID":"4f065c68-65f3-4014-8417-31619fc4d975","Type":"ContainerStarted","Data":"7ab99128454fb298b7c6558a302423fd5b337958b6769eee89514447523243f3"}
May 21 17:43:01.605047 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.605031    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c" event={"ID":"4f065c68-65f3-4014-8417-31619fc4d975","Type":"ContainerStarted","Data":"66d79fb2e9bb4c04ac15a6e3ff49bdd317c6e6e9b151dfab418b348b942c835f"}
May 21 17:43:01.605047 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.605042    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c" event={"ID":"4f065c68-65f3-4014-8417-31619fc4d975","Type":"ContainerStarted","Data":"296c11e60096bd7db5031fa66e62d97e86cc3aa422fe5766be98e4c1b93f6072"}
May 21 17:43:01.605150 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.605052    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c" event={"ID":"4f065c68-65f3-4014-8417-31619fc4d975","Type":"ContainerStarted","Data":"fc78f46b5b8d9269eb2c3a521654909905e7bd012a8b7767dfefef7d4ce16c43"}
May 21 17:43:01.605150 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.605060    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c" event={"ID":"4f065c68-65f3-4014-8417-31619fc4d975","Type":"ContainerDied","Data":"d3502caa6c9aadf3f1449c66aaf03a3989fb639ed6c6fb47bd63fe89b6281ac2"}
May 21 17:43:01.605150 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.605070    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c" event={"ID":"4f065c68-65f3-4014-8417-31619fc4d975","Type":"ContainerStarted","Data":"30386b392f8b78f2d52ee8bce0626cfb25e59873912fd6e1dac08bf7c42d834e"}
May 21 17:43:01.606256 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.606238    2558 generic.go:358] "Generic (PLEG): container finished" podID="22e5cf05-8688-49c7-8390-94b6030b614c" containerID="44f1c2983e83e6574d7b9dc0ab5bc6bd0e4cd5606272b95dd2f07c34cbedb68c" exitCode=0
May 21 17:43:01.606329 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.606302    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtxx2" event={"ID":"22e5cf05-8688-49c7-8390-94b6030b614c","Type":"ContainerDied","Data":"44f1c2983e83e6574d7b9dc0ab5bc6bd0e4cd5606272b95dd2f07c34cbedb68c"}
May 21 17:43:01.607635 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.607530    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp" event={"ID":"9c81a540-d2cd-41e1-bb7f-e474458ece54","Type":"ContainerStarted","Data":"238d702bfdc43be56d80db7e7faa5c234f081ab044655a041e2f41effff18d5a"}
May 21 17:43:01.608720 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.608698    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zcbrp" event={"ID":"511e916d-ee83-4b83-a9e4-811ff6092f9a","Type":"ContainerStarted","Data":"f4fab748b2dc0bf20ebd8111939bbab826da42256e63b8ecb83c966de2d3c661"}
May 21 17:43:01.608938 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.608923    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-brblc"
May 21 17:43:01.609392 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.609377    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-brblc"
May 21 17:43:01.630355 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.630311    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-mkh56" podStartSLOduration=4.448961132 podStartE2EDuration="21.630299643s" podCreationTimestamp="2026-05-21 17:42:40 +0000 UTC" firstStartedPulling="2026-05-21 17:42:43.261407289 +0000 UTC m=+3.371800262" lastFinishedPulling="2026-05-21 17:43:00.442745815 +0000 UTC m=+20.553138773" observedRunningTime="2026-05-21 17:43:01.616982464 +0000 UTC m=+21.727375445" watchObservedRunningTime="2026-05-21 17:43:01.630299643 +0000 UTC m=+21.740692623"
May 21 17:43:01.640407 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.640368    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zcbrp" podStartSLOduration=4.461000232 podStartE2EDuration="21.640360819s" podCreationTimestamp="2026-05-21 17:42:40 +0000 UTC" firstStartedPulling="2026-05-21 17:42:43.263030456 +0000 UTC m=+3.373423425" lastFinishedPulling="2026-05-21 17:43:00.442391047 +0000 UTC m=+20.552784012" observedRunningTime="2026-05-21 17:43:01.640160494 +0000 UTC m=+21.750553465" watchObservedRunningTime="2026-05-21 17:43:01.640360819 +0000 UTC m=+21.750753799"
May 21 17:43:01.674050 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:01.674014    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-h9lc4" podStartSLOduration=1.6740020549999999 podStartE2EDuration="1.674002055s" podCreationTimestamp="2026-05-21 17:43:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 17:43:01.651875521 +0000 UTC m=+21.762268508" watchObservedRunningTime="2026-05-21 17:43:01.674002055 +0000 UTC m=+21.784395045"
May 21 17:43:02.030474 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:02.030323    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 21 17:43:02.447418 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:02.447317    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-21T17:43:02.030470204Z","UUID":"cd945f1c-f919-4795-b0d2-439aec3a6152","Handler":null,"Name":"","Endpoint":""}
May 21 17:43:02.450553 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:02.450532    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 21 17:43:02.450553 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:02.450563    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 21 17:43:02.502898 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:02.502870    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:43:02.503060 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:02.502880    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:43:02.503060 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:02.502995    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:43:02.503159 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:02.503076    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:43:02.612763 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:02.612723    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rmh5q" event={"ID":"2c0d8636-3bdd-479b-a341-b7a8094b9641","Type":"ContainerStarted","Data":"6e19061581658998ff8647369737434a4e435dffdf9c25607d4887b73c5ffea7"}
May 21 17:43:02.615145 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:02.615054    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp" event={"ID":"9c81a540-d2cd-41e1-bb7f-e474458ece54","Type":"ContainerStarted","Data":"c08f51729a6c7f7ce05f157f683748f13e86e6a88731802375c8966e9d67b295"}
May 21 17:43:02.635202 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:02.635156    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rmh5q" podStartSLOduration=5.500769014 podStartE2EDuration="22.635136862s" podCreationTimestamp="2026-05-21 17:42:40 +0000 UTC" firstStartedPulling="2026-05-21 17:42:43.260054078 +0000 UTC m=+3.370447037" lastFinishedPulling="2026-05-21 17:43:00.394421913 +0000 UTC m=+20.504814885" observedRunningTime="2026-05-21 17:43:02.635021347 +0000 UTC m=+22.745414326" watchObservedRunningTime="2026-05-21 17:43:02.635136862 +0000 UTC m=+22.745529843"
May 21 17:43:03.618529 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:03.618433    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp" event={"ID":"9c81a540-d2cd-41e1-bb7f-e474458ece54","Type":"ContainerStarted","Data":"816055773eafe3ddc25f02ea8cc8ce86dc214c89eb9f511743651c215890c809"}
May 21 17:43:03.620891 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:03.620872    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 17:43:03.621183 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:03.621156    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c" event={"ID":"4f065c68-65f3-4014-8417-31619fc4d975","Type":"ContainerStarted","Data":"66651a241b0439ce8351242228562950dab7ddffd130a06f22ae4817af90954c"}
May 21 17:43:03.635671 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:03.635629    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2bnp" podStartSLOduration=3.6781214330000003 podStartE2EDuration="23.635617702s" podCreationTimestamp="2026-05-21 17:42:40 +0000 UTC" firstStartedPulling="2026-05-21 17:42:43.263886288 +0000 UTC m=+3.374279263" lastFinishedPulling="2026-05-21 17:43:03.221382574 +0000 UTC m=+23.331775532" observedRunningTime="2026-05-21 17:43:03.635255118 +0000 UTC m=+23.745648094" watchObservedRunningTime="2026-05-21 17:43:03.635617702 +0000 UTC m=+23.746010683"
May 21 17:43:04.502421 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:04.502383    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:43:04.502605 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:04.502534    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:43:04.502808 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:04.502383    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:43:04.502939 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:04.502908    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:43:05.627796 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:05.627564    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 17:43:05.628485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:05.628213    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c" event={"ID":"4f065c68-65f3-4014-8417-31619fc4d975","Type":"ContainerStarted","Data":"381758953a0057a38c55fb64795465d997c197c6befa7571b9cb8333afb196ad"}
May 21 17:43:05.628622 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:05.628602    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:43:05.628852 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:05.628826    2558 scope.go:117] "RemoveContainer" containerID="d3502caa6c9aadf3f1449c66aaf03a3989fb639ed6c6fb47bd63fe89b6281ac2"
May 21 17:43:05.646175 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:05.646133    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:43:06.004769 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.004694    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-5qhm9"]
May 21 17:43:06.007243 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.007222    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:06.007345 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:06.007303    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qhm9" podUID="dcb17a36-6e79-49f3-a00d-840467406c82"
May 21 17:43:06.076385 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.076340    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret\") pod \"global-pull-secret-syncer-5qhm9\" (UID: \"dcb17a36-6e79-49f3-a00d-840467406c82\") " pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:06.076569 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.076480    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dcb17a36-6e79-49f3-a00d-840467406c82-kubelet-config\") pod \"global-pull-secret-syncer-5qhm9\" (UID: \"dcb17a36-6e79-49f3-a00d-840467406c82\") " pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:06.076569 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.076534    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dcb17a36-6e79-49f3-a00d-840467406c82-dbus\") pod \"global-pull-secret-syncer-5qhm9\" (UID: \"dcb17a36-6e79-49f3-a00d-840467406c82\") " pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:06.177614 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.177581    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dcb17a36-6e79-49f3-a00d-840467406c82-kubelet-config\") pod \"global-pull-secret-syncer-5qhm9\" (UID: \"dcb17a36-6e79-49f3-a00d-840467406c82\") " pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:06.177752 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.177625    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dcb17a36-6e79-49f3-a00d-840467406c82-dbus\") pod \"global-pull-secret-syncer-5qhm9\" (UID: \"dcb17a36-6e79-49f3-a00d-840467406c82\") " pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:06.177752 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.177681    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret\") pod \"global-pull-secret-syncer-5qhm9\" (UID: \"dcb17a36-6e79-49f3-a00d-840467406c82\") " pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:06.177752 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.177722    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dcb17a36-6e79-49f3-a00d-840467406c82-kubelet-config\") pod \"global-pull-secret-syncer-5qhm9\" (UID: \"dcb17a36-6e79-49f3-a00d-840467406c82\") " pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:06.177894 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:06.177854    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 21 17:43:06.177931 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:06.177918    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret podName:dcb17a36-6e79-49f3-a00d-840467406c82 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:06.677899955 +0000 UTC m=+26.788292913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret") pod "global-pull-secret-syncer-5qhm9" (UID: "dcb17a36-6e79-49f3-a00d-840467406c82") : object "kube-system"/"original-pull-secret" not registered
May 21 17:43:06.177977 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.177936    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dcb17a36-6e79-49f3-a00d-840467406c82-dbus\") pod \"global-pull-secret-syncer-5qhm9\" (UID: \"dcb17a36-6e79-49f3-a00d-840467406c82\") " pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:06.503469 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.503385    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:43:06.503636 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:06.503511    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:43:06.503636 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.503616    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:43:06.503770 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:06.503746    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:43:06.635195 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.635140    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 17:43:06.635653 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.635579    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c" event={"ID":"4f065c68-65f3-4014-8417-31619fc4d975","Type":"ContainerStarted","Data":"4479ea10fcbc44704a388ddcae5e3f7839d9b794d67c90256dafe481a2e2b514"}
May 21 17:43:06.636063 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.636027    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:43:06.636180 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.636152    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:43:06.661004 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.655702    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:43:06.664598 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.664547    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c" podStartSLOduration=9.408127383 podStartE2EDuration="26.664530449s" podCreationTimestamp="2026-05-21 17:42:40 +0000 UTC" firstStartedPulling="2026-05-21 17:42:43.255417563 +0000 UTC m=+3.365810521" lastFinishedPulling="2026-05-21 17:43:00.511820612 +0000 UTC m=+20.622213587" observedRunningTime="2026-05-21 17:43:06.663550943 +0000 UTC m=+26.773943924" watchObservedRunningTime="2026-05-21 17:43:06.664530449 +0000 UTC m=+26.774923432"
May 21 17:43:06.681011 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:06.680980    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret\") pod \"global-pull-secret-syncer-5qhm9\" (UID: \"dcb17a36-6e79-49f3-a00d-840467406c82\") " pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:06.681162 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:06.681147    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 21 17:43:06.681238 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:06.681225    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret podName:dcb17a36-6e79-49f3-a00d-840467406c82 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:07.681205246 +0000 UTC m=+27.791598228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret") pod "global-pull-secret-syncer-5qhm9" (UID: "dcb17a36-6e79-49f3-a00d-840467406c82") : object "kube-system"/"original-pull-secret" not registered
May 21 17:43:07.502805 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:07.502769    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:07.502918 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:07.502890    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qhm9" podUID="dcb17a36-6e79-49f3-a00d-840467406c82"
May 21 17:43:07.641194 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:07.640799    2558 generic.go:358] "Generic (PLEG): container finished" podID="22e5cf05-8688-49c7-8390-94b6030b614c" containerID="280cab0f97d5947fb0d794990405c69ba793ae905cd8a84df3561ff5dd31d0cf" exitCode=0
May 21 17:43:07.641194 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:07.640898    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtxx2" event={"ID":"22e5cf05-8688-49c7-8390-94b6030b614c","Type":"ContainerDied","Data":"280cab0f97d5947fb0d794990405c69ba793ae905cd8a84df3561ff5dd31d0cf"}
May 21 17:43:07.648258 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:07.648238    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5qhm9"]
May 21 17:43:07.648350 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:07.648310    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:07.648405 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:07.648383    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qhm9" podUID="dcb17a36-6e79-49f3-a00d-840467406c82"
May 21 17:43:07.651429 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:07.651406    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-brzgc"]
May 21 17:43:07.651545 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:07.651487    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:43:07.651636 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:07.651613    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:43:07.652023 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:07.651996    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sk2jf"]
May 21 17:43:07.652168 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:07.652070    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:43:07.652229 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:07.652166    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:43:07.687753 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:07.687696    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret\") pod \"global-pull-secret-syncer-5qhm9\" (UID: \"dcb17a36-6e79-49f3-a00d-840467406c82\") " pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:07.687901 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:07.687846    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 21 17:43:07.687944 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:07.687909    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret podName:dcb17a36-6e79-49f3-a00d-840467406c82 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:09.68789214 +0000 UTC m=+29.798285105 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret") pod "global-pull-secret-syncer-5qhm9" (UID: "dcb17a36-6e79-49f3-a00d-840467406c82") : object "kube-system"/"original-pull-secret" not registered
May 21 17:43:08.645257 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:08.645223    2558 generic.go:358] "Generic (PLEG): container finished" podID="22e5cf05-8688-49c7-8390-94b6030b614c" containerID="86d11222a8d72060071682efdfb0c2dece2ab5445567152640e72b87e8417937" exitCode=0
May 21 17:43:08.645717 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:08.645304    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtxx2" event={"ID":"22e5cf05-8688-49c7-8390-94b6030b614c","Type":"ContainerDied","Data":"86d11222a8d72060071682efdfb0c2dece2ab5445567152640e72b87e8417937"}
May 21 17:43:09.503367 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:09.503343    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:43:09.503367 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:09.503358    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:09.503519 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:09.503469    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:43:09.503519 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:09.503508    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qhm9" podUID="dcb17a36-6e79-49f3-a00d-840467406c82"
May 21 17:43:09.503601 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:09.503548    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:43:09.503694 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:09.503667    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:43:09.649224 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:09.649141    2558 generic.go:358] "Generic (PLEG): container finished" podID="22e5cf05-8688-49c7-8390-94b6030b614c" containerID="c681e677bc233473fa136ce53544cb352b898924a88aede852dca482dc5f3b2f" exitCode=0
May 21 17:43:09.649224 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:09.649181    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtxx2" event={"ID":"22e5cf05-8688-49c7-8390-94b6030b614c","Type":"ContainerDied","Data":"c681e677bc233473fa136ce53544cb352b898924a88aede852dca482dc5f3b2f"}
May 21 17:43:09.705032 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:09.704997    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret\") pod \"global-pull-secret-syncer-5qhm9\" (UID: \"dcb17a36-6e79-49f3-a00d-840467406c82\") " pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:09.705187 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:09.705154    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 21 17:43:09.705266 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:09.705240    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret podName:dcb17a36-6e79-49f3-a00d-840467406c82 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:13.705220392 +0000 UTC m=+33.815613352 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret") pod "global-pull-secret-syncer-5qhm9" (UID: "dcb17a36-6e79-49f3-a00d-840467406c82") : object "kube-system"/"original-pull-secret" not registered
May 21 17:43:11.503066 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:11.503032    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:43:11.503872 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:11.503076    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:43:11.503872 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:11.503077    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:11.503872 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:11.503155    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:43:11.503872 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:11.503261    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:43:11.503872 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:11.503327    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qhm9" podUID="dcb17a36-6e79-49f3-a00d-840467406c82"
May 21 17:43:13.502951 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:13.502918    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:13.503470 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:13.502934    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:43:13.503470 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:13.503037    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:43:13.503470 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:13.503039    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5qhm9" podUID="dcb17a36-6e79-49f3-a00d-840467406c82"
May 21 17:43:13.503470 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:13.503100    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-brzgc" podUID="6db0adc0-a33d-4eca-988e-073a4bd19f80"
May 21 17:43:13.503470 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:13.503197    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:43:13.734821 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:13.734768    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret\") pod \"global-pull-secret-syncer-5qhm9\" (UID: \"dcb17a36-6e79-49f3-a00d-840467406c82\") " pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:13.734993 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:13.734916    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 21 17:43:13.735046 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:13.735000    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret podName:dcb17a36-6e79-49f3-a00d-840467406c82 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:21.734977824 +0000 UTC m=+41.845370804 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret") pod "global-pull-secret-syncer-5qhm9" (UID: "dcb17a36-6e79-49f3-a00d-840467406c82") : object "kube-system"/"original-pull-secret" not registered
May 21 17:43:14.138031 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.137993    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs\") pod \"network-metrics-daemon-sk2jf\" (UID: \"48a7b5cd-49ef-4177-83f1-80c264282bf9\") " pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:43:14.138210 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.138162    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 17:43:14.138281 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.138242    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs podName:48a7b5cd-49ef-4177-83f1-80c264282bf9 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:46.138222154 +0000 UTC m=+66.248615119 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs") pod "network-metrics-daemon-sk2jf" (UID: "48a7b5cd-49ef-4177-83f1-80c264282bf9") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 21 17:43:14.182937 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.182902    2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-125.ec2.internal" event="NodeReady"
May 21 17:43:14.183087 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.183063    2558 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 21 17:43:14.219310 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.219272    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"]
May 21 17:43:14.225014 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.224987    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"]
May 21 17:43:14.225184 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.225161    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:43:14.227439 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.227411    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-r7lth\""
May 21 17:43:14.227552 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.227407    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
May 21 17:43:14.227552 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.227461    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
May 21 17:43:14.228237 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.228222    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.230107 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.230047    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 21 17:43:14.230107 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.230050    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 21 17:43:14.230263 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.230182    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 21 17:43:14.230497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.230481    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qnr4m\""
May 21 17:43:14.234000 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.233979    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-phkwf"]
May 21 17:43:14.237884 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.237859    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"]
May 21 17:43:14.238022 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.237893    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"]
May 21 17:43:14.238022 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.237928    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 21 17:43:14.238235 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.238046    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-phkwf"
May 21 17:43:14.239931 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.239910    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dd2vd"]
May 21 17:43:14.240277 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.240261    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 21 17:43:14.240373 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.240351    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 21 17:43:14.240830 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.240811    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wq5rz\""
May 21 17:43:14.243195 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.243177    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:43:14.245358 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.245232    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 21 17:43:14.245881 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.245639    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5t9n8\""
May 21 17:43:14.245997 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.245976    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 21 17:43:14.246200 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.246179    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-phkwf"]
May 21 17:43:14.246335 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.246201    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 21 17:43:14.252453 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.252384    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dd2vd"]
May 21 17:43:14.338975 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.338940    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e505019e-ed7b-4564-861e-17d31ff967f5-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-gwldh\" (UID: \"e505019e-ed7b-4564-861e-17d31ff967f5\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:43:14.339147 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.339021    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a0046ef-374c-4543-9402-f60e026fcefa-installation-pull-secrets\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.339147 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.339066    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-bound-sa-token\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.339147 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.339092    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert\") pod \"ingress-canary-dd2vd\" (UID: \"ec348fe3-0beb-4db4-be82-ec7970b5c9e9\") " pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:43:14.339147 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.339127    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.339330 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.339163    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ceaa27ad-561f-481b-aa5d-59f9c8448199-config-volume\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:14.339330 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.339194    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72kj7\" (UniqueName: \"kubernetes.io/projected/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-kube-api-access-72kj7\") pod \"ingress-canary-dd2vd\" (UID: \"ec348fe3-0beb-4db4-be82-ec7970b5c9e9\") " pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:43:14.339330 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.339296    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:14.339330 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.339324    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ccmp\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-kube-api-access-5ccmp\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.339479 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.339362    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-gwldh\" (UID: \"e505019e-ed7b-4564-861e-17d31ff967f5\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:43:14.339479 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.339398    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a0046ef-374c-4543-9402-f60e026fcefa-ca-trust-extracted\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.339479 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.339466    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ceaa27ad-561f-481b-aa5d-59f9c8448199-tmp-dir\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:14.339581 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.339494    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qknv4\" (UniqueName: \"kubernetes.io/projected/ceaa27ad-561f-481b-aa5d-59f9c8448199-kube-api-access-qknv4\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:14.339581 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.339523    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a0046ef-374c-4543-9402-f60e026fcefa-trusted-ca\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.339581 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.339572    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tzhfd\" (UniqueName: \"kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd\") pod \"network-check-target-brzgc\" (UID: \"6db0adc0-a33d-4eca-988e-073a4bd19f80\") " pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:43:14.339702 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.339641    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0a0046ef-374c-4543-9402-f60e026fcefa-image-registry-private-configuration\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.339702 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.339677    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a0046ef-374c-4543-9402-f60e026fcefa-registry-certificates\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.339806 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.339704    2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 21 17:43:14.339806 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.339722    2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 21 17:43:14.339806 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.339736    2558 projected.go:194] Error preparing data for projected volume kube-api-access-tzhfd for pod openshift-network-diagnostics/network-check-target-brzgc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 17:43:14.339806 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.339802    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd podName:6db0adc0-a33d-4eca-988e-073a4bd19f80 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:46.339772743 +0000 UTC m=+66.450165706 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzhfd" (UniqueName: "kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd") pod "network-check-target-brzgc" (UID: "6db0adc0-a33d-4eca-988e-073a4bd19f80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 21 17:43:14.440018 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.439975    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a0046ef-374c-4543-9402-f60e026fcefa-trusted-ca\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.440206 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.440056    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0a0046ef-374c-4543-9402-f60e026fcefa-image-registry-private-configuration\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.440206 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.440087    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a0046ef-374c-4543-9402-f60e026fcefa-registry-certificates\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.440335 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.440215    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e505019e-ed7b-4564-861e-17d31ff967f5-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-gwldh\" (UID: \"e505019e-ed7b-4564-861e-17d31ff967f5\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:43:14.440335 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.440274    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a0046ef-374c-4543-9402-f60e026fcefa-installation-pull-secrets\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.440335 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.440317    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-bound-sa-token\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.440482 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.440354    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert\") pod \"ingress-canary-dd2vd\" (UID: \"ec348fe3-0beb-4db4-be82-ec7970b5c9e9\") " pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:43:14.440482 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.440392    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.440482 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.440427    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ceaa27ad-561f-481b-aa5d-59f9c8448199-config-volume\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:14.440482 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.440465    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-72kj7\" (UniqueName: \"kubernetes.io/projected/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-kube-api-access-72kj7\") pod \"ingress-canary-dd2vd\" (UID: \"ec348fe3-0beb-4db4-be82-ec7970b5c9e9\") " pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:43:14.440671 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.440527    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:14.440671 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.440588    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5ccmp\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-kube-api-access-5ccmp\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.440671 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.440640    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-gwldh\" (UID: \"e505019e-ed7b-4564-861e-17d31ff967f5\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:43:14.440839 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.440669    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a0046ef-374c-4543-9402-f60e026fcefa-ca-trust-extracted\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.440839 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.440718    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ceaa27ad-561f-481b-aa5d-59f9c8448199-tmp-dir\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:14.440839 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.440743    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qknv4\" (UniqueName: \"kubernetes.io/projected/ceaa27ad-561f-481b-aa5d-59f9c8448199-kube-api-access-qknv4\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:14.441037 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.440950    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a0046ef-374c-4543-9402-f60e026fcefa-registry-certificates\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.441390 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.441360    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 17:43:14.441506 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.441414    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a0046ef-374c-4543-9402-f60e026fcefa-trusted-ca\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.441506 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.441432    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls podName:ceaa27ad-561f-481b-aa5d-59f9c8448199 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:14.941413656 +0000 UTC m=+35.051806628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls") pod "dns-default-phkwf" (UID: "ceaa27ad-561f-481b-aa5d-59f9c8448199") : secret "dns-default-metrics-tls" not found
May 21 17:43:14.441506 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.441482    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ceaa27ad-561f-481b-aa5d-59f9c8448199-config-volume\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:14.441679 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.441514    2558 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 17:43:14.441679 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.441564    2558 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 21 17:43:14.441679 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.441571    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert podName:ec348fe3-0beb-4db4-be82-ec7970b5c9e9 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:14.941555314 +0000 UTC m=+35.051948282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert") pod "ingress-canary-dd2vd" (UID: "ec348fe3-0beb-4db4-be82-ec7970b5c9e9") : secret "canary-serving-cert" not found
May 21 17:43:14.441679 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.441584    2558 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f6c48bf8c-dlzc8: secret "image-registry-tls" not found
May 21 17:43:14.441679 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.441632    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls podName:0a0046ef-374c-4543-9402-f60e026fcefa nodeName:}" failed. No retries permitted until 2026-05-21 17:43:14.941618386 +0000 UTC m=+35.052011365 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls") pod "image-registry-6f6c48bf8c-dlzc8" (UID: "0a0046ef-374c-4543-9402-f60e026fcefa") : secret "image-registry-tls" not found
May 21 17:43:14.441679 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.441651    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 21 17:43:14.442013 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.441688    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert podName:e505019e-ed7b-4564-861e-17d31ff967f5 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:14.941676717 +0000 UTC m=+35.052069688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-gwldh" (UID: "e505019e-ed7b-4564-861e-17d31ff967f5") : secret "networking-console-plugin-cert" not found
May 21 17:43:14.442013 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.441770    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e505019e-ed7b-4564-861e-17d31ff967f5-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-gwldh\" (UID: \"e505019e-ed7b-4564-861e-17d31ff967f5\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:43:14.442013 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.441937    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ceaa27ad-561f-481b-aa5d-59f9c8448199-tmp-dir\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:14.442167 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.442084    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a0046ef-374c-4543-9402-f60e026fcefa-ca-trust-extracted\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.444858 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.444837    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a0046ef-374c-4543-9402-f60e026fcefa-installation-pull-secrets\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.444858 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.444851    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0a0046ef-374c-4543-9402-f60e026fcefa-image-registry-private-configuration\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.449690 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.449660    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72kj7\" (UniqueName: \"kubernetes.io/projected/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-kube-api-access-72kj7\") pod \"ingress-canary-dd2vd\" (UID: \"ec348fe3-0beb-4db4-be82-ec7970b5c9e9\") " pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:43:14.449690 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.449673    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qknv4\" (UniqueName: \"kubernetes.io/projected/ceaa27ad-561f-481b-aa5d-59f9c8448199-kube-api-access-qknv4\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:14.450003 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.449691    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-bound-sa-token\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.450098 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.450084    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ccmp\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-kube-api-access-5ccmp\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.945113 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.945069    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert\") pod \"ingress-canary-dd2vd\" (UID: \"ec348fe3-0beb-4db4-be82-ec7970b5c9e9\") " pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:43:14.945676 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.945126    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:14.945676 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.945176    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:14.945676 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:14.945229    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-gwldh\" (UID: \"e505019e-ed7b-4564-861e-17d31ff967f5\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:43:14.945676 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.945253    2558 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 17:43:14.945676 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.945328    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert podName:ec348fe3-0beb-4db4-be82-ec7970b5c9e9 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:15.945307089 +0000 UTC m=+36.055700065 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert") pod "ingress-canary-dd2vd" (UID: "ec348fe3-0beb-4db4-be82-ec7970b5c9e9") : secret "canary-serving-cert" not found
May 21 17:43:14.945676 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.945330    2558 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 21 17:43:14.945676 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.945348    2558 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f6c48bf8c-dlzc8: secret "image-registry-tls" not found
May 21 17:43:14.945676 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.945352    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 21 17:43:14.945676 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.945370    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 17:43:14.945676 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.945390    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls podName:0a0046ef-374c-4543-9402-f60e026fcefa nodeName:}" failed. No retries permitted until 2026-05-21 17:43:15.945377383 +0000 UTC m=+36.055770342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls") pod "image-registry-6f6c48bf8c-dlzc8" (UID: "0a0046ef-374c-4543-9402-f60e026fcefa") : secret "image-registry-tls" not found
May 21 17:43:14.945676 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.945403    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert podName:e505019e-ed7b-4564-861e-17d31ff967f5 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:15.945397479 +0000 UTC m=+36.055790437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-gwldh" (UID: "e505019e-ed7b-4564-861e-17d31ff967f5") : secret "networking-console-plugin-cert" not found
May 21 17:43:14.945676 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:14.945414    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls podName:ceaa27ad-561f-481b-aa5d-59f9c8448199 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:15.945408892 +0000 UTC m=+36.055801850 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls") pod "dns-default-phkwf" (UID: "ceaa27ad-561f-481b-aa5d-59f9c8448199") : secret "dns-default-metrics-tls" not found
May 21 17:43:15.502833 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.502798    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:15.502833 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.502835    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:43:15.503083 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.502798    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:43:15.505234 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.505185    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 21 17:43:15.505234 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.505234    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 21 17:43:15.505696 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.505635    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 21 17:43:15.505696 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.505665    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kt2hm\""
May 21 17:43:15.506084 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.506065    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 21 17:43:15.506181 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.506171    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s4sh9\""
May 21 17:43:15.935001 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.934807    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86c4854848-gwvzm"]
May 21 17:43:15.950913 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.950885    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"]
May 21 17:43:15.951353 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.951040    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86c4854848-gwvzm"
May 21 17:43:15.952953 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.952925    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:15.953102 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.952995    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-gwldh\" (UID: \"e505019e-ed7b-4564-861e-17d31ff967f5\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:43:15.953102 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.953059    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
May 21 17:43:15.953102 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.953079    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert\") pod \"ingress-canary-dd2vd\" (UID: \"ec348fe3-0beb-4db4-be82-ec7970b5c9e9\") " pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:43:15.953272 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.953109    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:15.953272 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:15.953212    2558 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 21 17:43:15.953272 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:15.953225    2558 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f6c48bf8c-dlzc8: secret "image-registry-tls" not found
May 21 17:43:15.953272 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:15.953271    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls podName:0a0046ef-374c-4543-9402-f60e026fcefa nodeName:}" failed. No retries permitted until 2026-05-21 17:43:17.953255496 +0000 UTC m=+38.063648455 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls") pod "image-registry-6f6c48bf8c-dlzc8" (UID: "0a0046ef-374c-4543-9402-f60e026fcefa") : secret "image-registry-tls" not found
May 21 17:43:15.953462 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:15.953291    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 21 17:43:15.953462 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:15.953313    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 17:43:15.953462 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:15.953327    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert podName:e505019e-ed7b-4564-861e-17d31ff967f5 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:17.953316422 +0000 UTC m=+38.063709397 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-gwldh" (UID: "e505019e-ed7b-4564-861e-17d31ff967f5") : secret "networking-console-plugin-cert" not found
May 21 17:43:15.953462 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:15.953367    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls podName:ceaa27ad-561f-481b-aa5d-59f9c8448199 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:17.953348713 +0000 UTC m=+38.063741671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls") pod "dns-default-phkwf" (UID: "ceaa27ad-561f-481b-aa5d-59f9c8448199") : secret "dns-default-metrics-tls" not found
May 21 17:43:15.953462 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:15.953375    2558 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 17:43:15.953462 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:15.953402    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert podName:ec348fe3-0beb-4db4-be82-ec7970b5c9e9 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:17.953393235 +0000 UTC m=+38.063786209 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert") pod "ingress-canary-dd2vd" (UID: "ec348fe3-0beb-4db4-be82-ec7970b5c9e9") : secret "canary-serving-cert" not found
May 21 17:43:15.953771 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.953753    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
May 21 17:43:15.953853 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.953810    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-l5lnw\""
May 21 17:43:15.953853 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.953831    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
May 21 17:43:15.953958 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.953853    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
May 21 17:43:15.977045 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.977024    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86c4854848-gwvzm"]
May 21 17:43:15.977149 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.977050    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"]
May 21 17:43:15.977149 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.977065    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"]
May 21 17:43:15.977149 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.977133    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:15.979142 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.979123    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
May 21 17:43:15.979260 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.979174    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
May 21 17:43:15.979260 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.979182    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
May 21 17:43:15.979260 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:15.979243    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
May 21 17:43:16.003528 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.003450    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"]
May 21 17:43:16.003649 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.003555    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"
May 21 17:43:16.005597 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.005576    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
May 21 17:43:16.053968 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.053948    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6cecd918-46ce-482d-8468-6b0157a1da42-ca\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.054053 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.053982    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6cecd918-46ce-482d-8468-6b0157a1da42-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.054053 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.054003    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d469384a-983c-48c8-8d28-6f6ae5a22c4f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-86c4854848-gwvzm\" (UID: \"d469384a-983c-48c8-8d28-6f6ae5a22c4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86c4854848-gwvzm"
May 21 17:43:16.054127 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.054050    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/6cecd918-46ce-482d-8468-6b0157a1da42-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.054127 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.054087    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6cecd918-46ce-482d-8468-6b0157a1da42-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.054127 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.054121    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzbxk\" (UniqueName: \"kubernetes.io/projected/d469384a-983c-48c8-8d28-6f6ae5a22c4f-kube-api-access-jzbxk\") pod \"managed-serviceaccount-addon-agent-86c4854848-gwvzm\" (UID: \"d469384a-983c-48c8-8d28-6f6ae5a22c4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86c4854848-gwvzm"
May 21 17:43:16.054214 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.054200    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6cecd918-46ce-482d-8468-6b0157a1da42-hub\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.054247 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.054223    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzt79\" (UniqueName: \"kubernetes.io/projected/6cecd918-46ce-482d-8468-6b0157a1da42-kube-api-access-mzt79\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.155256 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.155225    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6cecd918-46ce-482d-8468-6b0157a1da42-ca\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.155379 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.155269    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/025463ef-f6b2-4ae5-9a5e-44831e3ede8c-klusterlet-config\") pod \"klusterlet-addon-workmgr-6985688c57-zqn5h\" (UID: \"025463ef-f6b2-4ae5-9a5e-44831e3ede8c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"
May 21 17:43:16.155379 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.155297    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d469384a-983c-48c8-8d28-6f6ae5a22c4f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-86c4854848-gwvzm\" (UID: \"d469384a-983c-48c8-8d28-6f6ae5a22c4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86c4854848-gwvzm"
May 21 17:43:16.155379 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.155345    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/6cecd918-46ce-482d-8468-6b0157a1da42-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.155545 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.155452    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6cecd918-46ce-482d-8468-6b0157a1da42-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.155545 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.155489    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jzbxk\" (UniqueName: \"kubernetes.io/projected/d469384a-983c-48c8-8d28-6f6ae5a22c4f-kube-api-access-jzbxk\") pod \"managed-serviceaccount-addon-agent-86c4854848-gwvzm\" (UID: \"d469384a-983c-48c8-8d28-6f6ae5a22c4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86c4854848-gwvzm"
May 21 17:43:16.155545 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.155525    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mzt79\" (UniqueName: \"kubernetes.io/projected/6cecd918-46ce-482d-8468-6b0157a1da42-kube-api-access-mzt79\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.155692 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.155544    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6cecd918-46ce-482d-8468-6b0157a1da42-hub\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.155692 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.155597    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxpfn\" (UniqueName: \"kubernetes.io/projected/025463ef-f6b2-4ae5-9a5e-44831e3ede8c-kube-api-access-xxpfn\") pod \"klusterlet-addon-workmgr-6985688c57-zqn5h\" (UID: \"025463ef-f6b2-4ae5-9a5e-44831e3ede8c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"
May 21 17:43:16.155692 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.155659    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/025463ef-f6b2-4ae5-9a5e-44831e3ede8c-tmp\") pod \"klusterlet-addon-workmgr-6985688c57-zqn5h\" (UID: \"025463ef-f6b2-4ae5-9a5e-44831e3ede8c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"
May 21 17:43:16.155884 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.155694    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6cecd918-46ce-482d-8468-6b0157a1da42-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.156204 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.156168    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/6cecd918-46ce-482d-8468-6b0157a1da42-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.158423 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.158397    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6cecd918-46ce-482d-8468-6b0157a1da42-ca\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.158550 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.158465    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6cecd918-46ce-482d-8468-6b0157a1da42-hub\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.158616 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.158561    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6cecd918-46ce-482d-8468-6b0157a1da42-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.158616 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.158604    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6cecd918-46ce-482d-8468-6b0157a1da42-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.158770 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.158754    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d469384a-983c-48c8-8d28-6f6ae5a22c4f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-86c4854848-gwvzm\" (UID: \"d469384a-983c-48c8-8d28-6f6ae5a22c4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86c4854848-gwvzm"
May 21 17:43:16.162624 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.162597    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzbxk\" (UniqueName: \"kubernetes.io/projected/d469384a-983c-48c8-8d28-6f6ae5a22c4f-kube-api-access-jzbxk\") pod \"managed-serviceaccount-addon-agent-86c4854848-gwvzm\" (UID: \"d469384a-983c-48c8-8d28-6f6ae5a22c4f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86c4854848-gwvzm"
May 21 17:43:16.163403 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.163382    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzt79\" (UniqueName: \"kubernetes.io/projected/6cecd918-46ce-482d-8468-6b0157a1da42-kube-api-access-mzt79\") pod \"cluster-proxy-proxy-agent-5d565474b5-79sh7\" (UID: \"6cecd918-46ce-482d-8468-6b0157a1da42\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.257163 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.257117    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xxpfn\" (UniqueName: \"kubernetes.io/projected/025463ef-f6b2-4ae5-9a5e-44831e3ede8c-kube-api-access-xxpfn\") pod \"klusterlet-addon-workmgr-6985688c57-zqn5h\" (UID: \"025463ef-f6b2-4ae5-9a5e-44831e3ede8c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"
May 21 17:43:16.257304 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.257188    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/025463ef-f6b2-4ae5-9a5e-44831e3ede8c-tmp\") pod \"klusterlet-addon-workmgr-6985688c57-zqn5h\" (UID: \"025463ef-f6b2-4ae5-9a5e-44831e3ede8c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"
May 21 17:43:16.257304 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.257299    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/025463ef-f6b2-4ae5-9a5e-44831e3ede8c-klusterlet-config\") pod \"klusterlet-addon-workmgr-6985688c57-zqn5h\" (UID: \"025463ef-f6b2-4ae5-9a5e-44831e3ede8c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"
May 21 17:43:16.260810 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.258315    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/025463ef-f6b2-4ae5-9a5e-44831e3ede8c-tmp\") pod \"klusterlet-addon-workmgr-6985688c57-zqn5h\" (UID: \"025463ef-f6b2-4ae5-9a5e-44831e3ede8c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"
May 21 17:43:16.265212 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.265183    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/025463ef-f6b2-4ae5-9a5e-44831e3ede8c-klusterlet-config\") pod \"klusterlet-addon-workmgr-6985688c57-zqn5h\" (UID: \"025463ef-f6b2-4ae5-9a5e-44831e3ede8c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"
May 21 17:43:16.265762 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.265741    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxpfn\" (UniqueName: \"kubernetes.io/projected/025463ef-f6b2-4ae5-9a5e-44831e3ede8c-kube-api-access-xxpfn\") pod \"klusterlet-addon-workmgr-6985688c57-zqn5h\" (UID: \"025463ef-f6b2-4ae5-9a5e-44831e3ede8c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"
May 21 17:43:16.272552 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.272535    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86c4854848-gwvzm"
May 21 17:43:16.285467 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.285443    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:43:16.328423 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.328382    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"
May 21 17:43:16.436111 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.436084    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"]
May 21 17:43:16.442453 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.442429    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86c4854848-gwvzm"]
May 21 17:43:16.451825 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:43:16.451796    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd469384a_983c_48c8_8d28_6f6ae5a22c4f.slice/crio-cf69b62fd87a735b664d21591a1498724bbec5c72a159ec38e49bfb9c31c3c3f WatchSource:0}: Error finding container cf69b62fd87a735b664d21591a1498724bbec5c72a159ec38e49bfb9c31c3c3f: Status 404 returned error can't find the container with id cf69b62fd87a735b664d21591a1498724bbec5c72a159ec38e49bfb9c31c3c3f
May 21 17:43:16.452045 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:43:16.452024    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6cecd918_46ce_482d_8468_6b0157a1da42.slice/crio-815f115373d0517d62f15f0482e3934272f9f8724fe63da55ae485b773d818a6 WatchSource:0}: Error finding container 815f115373d0517d62f15f0482e3934272f9f8724fe63da55ae485b773d818a6: Status 404 returned error can't find the container with id 815f115373d0517d62f15f0482e3934272f9f8724fe63da55ae485b773d818a6
May 21 17:43:16.552579 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.552547    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"]
May 21 17:43:16.556271 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:43:16.556231    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod025463ef_f6b2_4ae5_9a5e_44831e3ede8c.slice/crio-16ee5ca2d1027cd825532dec6ab3237774034560b27e34527b88a594afa157bd WatchSource:0}: Error finding container 16ee5ca2d1027cd825532dec6ab3237774034560b27e34527b88a594afa157bd: Status 404 returned error can't find the container with id 16ee5ca2d1027cd825532dec6ab3237774034560b27e34527b88a594afa157bd
May 21 17:43:16.664010 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.663977    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86c4854848-gwvzm" event={"ID":"d469384a-983c-48c8-8d28-6f6ae5a22c4f","Type":"ContainerStarted","Data":"cf69b62fd87a735b664d21591a1498724bbec5c72a159ec38e49bfb9c31c3c3f"}
May 21 17:43:16.666223 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.666196    2558 generic.go:358] "Generic (PLEG): container finished" podID="22e5cf05-8688-49c7-8390-94b6030b614c" containerID="51cbf9ed7bc8cb5c5a339aabc0d940e264b11094fb82c00e87fe3aa965e45755" exitCode=0
May 21 17:43:16.666339 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.666264    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtxx2" event={"ID":"22e5cf05-8688-49c7-8390-94b6030b614c","Type":"ContainerDied","Data":"51cbf9ed7bc8cb5c5a339aabc0d940e264b11094fb82c00e87fe3aa965e45755"}
May 21 17:43:16.667236 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.667215    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h" event={"ID":"025463ef-f6b2-4ae5-9a5e-44831e3ede8c","Type":"ContainerStarted","Data":"16ee5ca2d1027cd825532dec6ab3237774034560b27e34527b88a594afa157bd"}
May 21 17:43:16.668206 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:16.668189    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7" event={"ID":"6cecd918-46ce-482d-8468-6b0157a1da42","Type":"ContainerStarted","Data":"815f115373d0517d62f15f0482e3934272f9f8724fe63da55ae485b773d818a6"}
May 21 17:43:17.678034 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:17.676939    2558 generic.go:358] "Generic (PLEG): container finished" podID="22e5cf05-8688-49c7-8390-94b6030b614c" containerID="9923446fdceee299924c81a121918c9eb1bbef595e462e1ed2ff0cd00886ec0b" exitCode=0
May 21 17:43:17.678034 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:17.677209    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtxx2" event={"ID":"22e5cf05-8688-49c7-8390-94b6030b614c","Type":"ContainerDied","Data":"9923446fdceee299924c81a121918c9eb1bbef595e462e1ed2ff0cd00886ec0b"}
May 21 17:43:17.973440 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:17.973366    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert\") pod \"ingress-canary-dd2vd\" (UID: \"ec348fe3-0beb-4db4-be82-ec7970b5c9e9\") " pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:43:17.973597 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:17.973521    2558 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 17:43:17.973597 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:17.973594    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert podName:ec348fe3-0beb-4db4-be82-ec7970b5c9e9 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:21.973574248 +0000 UTC m=+42.083967224 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert") pod "ingress-canary-dd2vd" (UID: "ec348fe3-0beb-4db4-be82-ec7970b5c9e9") : secret "canary-serving-cert" not found
May 21 17:43:17.973977 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:17.973955    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:17.974061 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:17.974045    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:17.974121 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:17.974092    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-gwldh\" (UID: \"e505019e-ed7b-4564-861e-17d31ff967f5\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:43:17.974235 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:17.974222    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 21 17:43:17.974299 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:17.974277    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert podName:e505019e-ed7b-4564-861e-17d31ff967f5 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:21.974262825 +0000 UTC m=+42.084655785 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-gwldh" (UID: "e505019e-ed7b-4564-861e-17d31ff967f5") : secret "networking-console-plugin-cert" not found
May 21 17:43:17.974356 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:17.974340    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 17:43:17.974404 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:17.974369    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls podName:ceaa27ad-561f-481b-aa5d-59f9c8448199 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:21.974359696 +0000 UTC m=+42.084752672 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls") pod "dns-default-phkwf" (UID: "ceaa27ad-561f-481b-aa5d-59f9c8448199") : secret "dns-default-metrics-tls" not found
May 21 17:43:17.974462 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:17.974432    2558 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 21 17:43:17.974462 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:17.974443    2558 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f6c48bf8c-dlzc8: secret "image-registry-tls" not found
May 21 17:43:17.974550 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:17.974487    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls podName:0a0046ef-374c-4543-9402-f60e026fcefa nodeName:}" failed. No retries permitted until 2026-05-21 17:43:21.974463612 +0000 UTC m=+42.084856570 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls") pod "image-registry-6f6c48bf8c-dlzc8" (UID: "0a0046ef-374c-4543-9402-f60e026fcefa") : secret "image-registry-tls" not found
May 21 17:43:18.686384 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:18.686345    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtxx2" event={"ID":"22e5cf05-8688-49c7-8390-94b6030b614c","Type":"ContainerStarted","Data":"904272e79c903d5575b1e40abb6f20f8a6eb60247d57b3526cfd4526d6285c37"}
May 21 17:43:18.708068 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:18.707576    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qtxx2" podStartSLOduration=5.822465079 podStartE2EDuration="38.707561899s" podCreationTimestamp="2026-05-21 17:42:40 +0000 UTC" firstStartedPulling="2026-05-21 17:42:43.254803634 +0000 UTC m=+3.365196592" lastFinishedPulling="2026-05-21 17:43:16.139900455 +0000 UTC m=+36.250293412" observedRunningTime="2026-05-21 17:43:18.706710802 +0000 UTC m=+38.817103801" watchObservedRunningTime="2026-05-21 17:43:18.707561899 +0000 UTC m=+38.817954878"
May 21 17:43:21.812422 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:21.812384    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret\") pod \"global-pull-secret-syncer-5qhm9\" (UID: \"dcb17a36-6e79-49f3-a00d-840467406c82\") " pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:21.815751 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:21.815728    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dcb17a36-6e79-49f3-a00d-840467406c82-original-pull-secret\") pod \"global-pull-secret-syncer-5qhm9\" (UID: \"dcb17a36-6e79-49f3-a00d-840467406c82\") " pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:22.014316 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:22.014267    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:22.014502 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:22.014331    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-gwldh\" (UID: \"e505019e-ed7b-4564-861e-17d31ff967f5\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:43:22.014502 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:22.014381    2558 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 21 17:43:22.014502 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:22.014398    2558 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f6c48bf8c-dlzc8: secret "image-registry-tls" not found
May 21 17:43:22.014502 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:22.014397    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert\") pod \"ingress-canary-dd2vd\" (UID: \"ec348fe3-0beb-4db4-be82-ec7970b5c9e9\") " pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:43:22.014502 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:22.014456    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls podName:0a0046ef-374c-4543-9402-f60e026fcefa nodeName:}" failed. No retries permitted until 2026-05-21 17:43:30.014438468 +0000 UTC m=+50.124831447 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls") pod "image-registry-6f6c48bf8c-dlzc8" (UID: "0a0046ef-374c-4543-9402-f60e026fcefa") : secret "image-registry-tls" not found
May 21 17:43:22.014502 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:22.014477    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:22.014502 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:22.014493    2558 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 17:43:22.014931 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:22.014504    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 21 17:43:22.014931 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:22.014530    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert podName:ec348fe3-0beb-4db4-be82-ec7970b5c9e9 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:30.014519166 +0000 UTC m=+50.124912138 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert") pod "ingress-canary-dd2vd" (UID: "ec348fe3-0beb-4db4-be82-ec7970b5c9e9") : secret "canary-serving-cert" not found
May 21 17:43:22.014931 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:22.014576    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 17:43:22.014931 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:22.014589    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert podName:e505019e-ed7b-4564-861e-17d31ff967f5 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:30.014570208 +0000 UTC m=+50.124963181 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-gwldh" (UID: "e505019e-ed7b-4564-861e-17d31ff967f5") : secret "networking-console-plugin-cert" not found
May 21 17:43:22.014931 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:22.014614    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls podName:ceaa27ad-561f-481b-aa5d-59f9c8448199 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:30.014596848 +0000 UTC m=+50.124989819 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls") pod "dns-default-phkwf" (UID: "ceaa27ad-561f-481b-aa5d-59f9c8448199") : secret "dns-default-metrics-tls" not found
May 21 17:43:22.115691 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:22.115610    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5qhm9"
May 21 17:43:24.151197 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:24.151173    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5qhm9"]
May 21 17:43:24.155697 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:43:24.155667    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcb17a36_6e79_49f3_a00d_840467406c82.slice/crio-2e90d21170f9f4389e0927d6b4c33525f6654904f4be6d17681f1543c953da01 WatchSource:0}: Error finding container 2e90d21170f9f4389e0927d6b4c33525f6654904f4be6d17681f1543c953da01: Status 404 returned error can't find the container with id 2e90d21170f9f4389e0927d6b4c33525f6654904f4be6d17681f1543c953da01
May 21 17:43:24.699072 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:24.699013    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7" event={"ID":"6cecd918-46ce-482d-8468-6b0157a1da42","Type":"ContainerStarted","Data":"7bce89a4f24a69a1be2f97e59520c250fa6342bc8a458301d094d162c2b106d8"}
May 21 17:43:24.700461 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:24.700428    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86c4854848-gwvzm" event={"ID":"d469384a-983c-48c8-8d28-6f6ae5a22c4f","Type":"ContainerStarted","Data":"0c25ca6b63367e55014c3bce3a28a77de9dc83c99c3db5e5a5848ef02a4c7d14"}
May 21 17:43:24.701489 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:24.701460    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5qhm9" event={"ID":"dcb17a36-6e79-49f3-a00d-840467406c82","Type":"ContainerStarted","Data":"2e90d21170f9f4389e0927d6b4c33525f6654904f4be6d17681f1543c953da01"}
May 21 17:43:24.702630 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:24.702608    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h" event={"ID":"025463ef-f6b2-4ae5-9a5e-44831e3ede8c","Type":"ContainerStarted","Data":"736be79119ae6d72ad84f9acc81c61b97e1b06d6c88d23f08707ba7178fbf6b8"}
May 21 17:43:24.702824 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:24.702808    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"
May 21 17:43:24.704272 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:24.704253    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"
May 21 17:43:24.716149 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:24.716106    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86c4854848-gwvzm" podStartSLOduration=2.151970545 podStartE2EDuration="9.716092219s" podCreationTimestamp="2026-05-21 17:43:15 +0000 UTC" firstStartedPulling="2026-05-21 17:43:16.454401728 +0000 UTC m=+36.564794686" lastFinishedPulling="2026-05-21 17:43:24.018523399 +0000 UTC m=+44.128916360" observedRunningTime="2026-05-21 17:43:24.715928116 +0000 UTC m=+44.826321095" watchObservedRunningTime="2026-05-21 17:43:24.716092219 +0000 UTC m=+44.826485201"
May 21 17:43:24.732603 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:24.732559    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h" podStartSLOduration=2.255666283 podStartE2EDuration="9.732548136s" podCreationTimestamp="2026-05-21 17:43:15 +0000 UTC" firstStartedPulling="2026-05-21 17:43:16.558090348 +0000 UTC m=+36.668483306" lastFinishedPulling="2026-05-21 17:43:24.0349722 +0000 UTC m=+44.145365159" observedRunningTime="2026-05-21 17:43:24.731905838 +0000 UTC m=+44.842298827" watchObservedRunningTime="2026-05-21 17:43:24.732548136 +0000 UTC m=+44.842941171"
May 21 17:43:27.712194 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:27.712158    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7" event={"ID":"6cecd918-46ce-482d-8468-6b0157a1da42","Type":"ContainerStarted","Data":"e23dd0b5d8ee37dc8c73be8d880b9e8ae39107a751d07b7cde23d476bad12443"}
May 21 17:43:28.716346 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:28.716312    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7" event={"ID":"6cecd918-46ce-482d-8468-6b0157a1da42","Type":"ContainerStarted","Data":"a7ee6ec9962151e58a129d14d5464c148b2dd230063bd3be244f093bdd85b4e3"}
May 21 17:43:28.735511 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:28.735461    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7" podStartSLOduration=3.126299945 podStartE2EDuration="13.735447658s" podCreationTimestamp="2026-05-21 17:43:15 +0000 UTC" firstStartedPulling="2026-05-21 17:43:16.45431006 +0000 UTC m=+36.564703018" lastFinishedPulling="2026-05-21 17:43:27.063457773 +0000 UTC m=+47.173850731" observedRunningTime="2026-05-21 17:43:28.733711599 +0000 UTC m=+48.844104579" watchObservedRunningTime="2026-05-21 17:43:28.735447658 +0000 UTC m=+48.845840639"
May 21 17:43:29.719889 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:29.719847    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5qhm9" event={"ID":"dcb17a36-6e79-49f3-a00d-840467406c82","Type":"ContainerStarted","Data":"f07d2c1c3d7b30aa2af4441acf05982a4f346426ee66f9467352f44533e6cf6a"}
May 21 17:43:29.733353 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:29.733309    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5qhm9" podStartSLOduration=19.986296689 podStartE2EDuration="24.733296075s" podCreationTimestamp="2026-05-21 17:43:05 +0000 UTC" firstStartedPulling="2026-05-21 17:43:24.157828165 +0000 UTC m=+44.268221126" lastFinishedPulling="2026-05-21 17:43:28.904827553 +0000 UTC m=+49.015220512" observedRunningTime="2026-05-21 17:43:29.732665205 +0000 UTC m=+49.843058186" watchObservedRunningTime="2026-05-21 17:43:29.733296075 +0000 UTC m=+49.843689054"
May 21 17:43:30.079640 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:30.079551    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert\") pod \"ingress-canary-dd2vd\" (UID: \"ec348fe3-0beb-4db4-be82-ec7970b5c9e9\") " pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:43:30.079640 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:30.079597    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:30.079640 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:30.079631    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:30.079986 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:30.079653    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-gwldh\" (UID: \"e505019e-ed7b-4564-861e-17d31ff967f5\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:43:30.079986 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:30.079682    2558 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 17:43:30.079986 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:30.079739    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 21 17:43:30.079986 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:30.079746    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert podName:ec348fe3-0beb-4db4-be82-ec7970b5c9e9 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:46.079732323 +0000 UTC m=+66.190125281 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert") pod "ingress-canary-dd2vd" (UID: "ec348fe3-0beb-4db4-be82-ec7970b5c9e9") : secret "canary-serving-cert" not found
May 21 17:43:30.079986 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:30.079738    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 17:43:30.079986 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:30.079802    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert podName:e505019e-ed7b-4564-861e-17d31ff967f5 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:46.07976834 +0000 UTC m=+66.190161298 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-gwldh" (UID: "e505019e-ed7b-4564-861e-17d31ff967f5") : secret "networking-console-plugin-cert" not found
May 21 17:43:30.079986 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:30.079846    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls podName:ceaa27ad-561f-481b-aa5d-59f9c8448199 nodeName:}" failed. No retries permitted until 2026-05-21 17:43:46.07983444 +0000 UTC m=+66.190227400 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls") pod "dns-default-phkwf" (UID: "ceaa27ad-561f-481b-aa5d-59f9c8448199") : secret "dns-default-metrics-tls" not found
May 21 17:43:30.079986 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:30.079779    2558 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 21 17:43:30.079986 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:30.079858    2558 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f6c48bf8c-dlzc8: secret "image-registry-tls" not found
May 21 17:43:30.079986 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:30.079885    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls podName:0a0046ef-374c-4543-9402-f60e026fcefa nodeName:}" failed. No retries permitted until 2026-05-21 17:43:46.07987682 +0000 UTC m=+66.190269778 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls") pod "image-registry-6f6c48bf8c-dlzc8" (UID: "0a0046ef-374c-4543-9402-f60e026fcefa") : secret "image-registry-tls" not found
May 21 17:43:38.655859 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:38.655827    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ht7c"
May 21 17:43:46.102768 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:46.102726    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:43:46.103205 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:46.102798    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:43:46.103205 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:46.102825    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-gwldh\" (UID: \"e505019e-ed7b-4564-861e-17d31ff967f5\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:43:46.103205 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:46.102860    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert\") pod \"ingress-canary-dd2vd\" (UID: \"ec348fe3-0beb-4db4-be82-ec7970b5c9e9\") " pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:43:46.103205 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:46.102896    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 17:43:46.103205 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:46.102955    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls podName:ceaa27ad-561f-481b-aa5d-59f9c8448199 nodeName:}" failed. No retries permitted until 2026-05-21 17:44:18.102941485 +0000 UTC m=+98.213334444 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls") pod "dns-default-phkwf" (UID: "ceaa27ad-561f-481b-aa5d-59f9c8448199") : secret "dns-default-metrics-tls" not found
May 21 17:43:46.103205 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:46.102981    2558 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 21 17:43:46.103205 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:46.103000    2558 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f6c48bf8c-dlzc8: secret "image-registry-tls" not found
May 21 17:43:46.103205 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:46.103023    2558 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 17:43:46.103205 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:46.103056    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls podName:0a0046ef-374c-4543-9402-f60e026fcefa nodeName:}" failed. No retries permitted until 2026-05-21 17:44:18.103037407 +0000 UTC m=+98.213430387 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls") pod "image-registry-6f6c48bf8c-dlzc8" (UID: "0a0046ef-374c-4543-9402-f60e026fcefa") : secret "image-registry-tls" not found
May 21 17:43:46.103205 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:46.103074    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert podName:ec348fe3-0beb-4db4-be82-ec7970b5c9e9 nodeName:}" failed. No retries permitted until 2026-05-21 17:44:18.103065501 +0000 UTC m=+98.213458462 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert") pod "ingress-canary-dd2vd" (UID: "ec348fe3-0beb-4db4-be82-ec7970b5c9e9") : secret "canary-serving-cert" not found
May 21 17:43:46.103205 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:46.103098    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 21 17:43:46.103205 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:46.103130    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert podName:e505019e-ed7b-4564-861e-17d31ff967f5 nodeName:}" failed. No retries permitted until 2026-05-21 17:44:18.103120982 +0000 UTC m=+98.213513940 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-gwldh" (UID: "e505019e-ed7b-4564-861e-17d31ff967f5") : secret "networking-console-plugin-cert" not found
May 21 17:43:46.203490 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:46.203455    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs\") pod \"network-metrics-daemon-sk2jf\" (UID: \"48a7b5cd-49ef-4177-83f1-80c264282bf9\") " pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:43:46.205839 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:46.205824    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 21 17:43:46.214654 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:46.214632    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 21 17:43:46.214734 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:43:46.214690    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs podName:48a7b5cd-49ef-4177-83f1-80c264282bf9 nodeName:}" failed. No retries permitted until 2026-05-21 17:44:50.214674736 +0000 UTC m=+130.325067693 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs") pod "network-metrics-daemon-sk2jf" (UID: "48a7b5cd-49ef-4177-83f1-80c264282bf9") : secret "metrics-daemon-secret" not found
May 21 17:43:46.405549 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:46.405467    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tzhfd\" (UniqueName: \"kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd\") pod \"network-check-target-brzgc\" (UID: \"6db0adc0-a33d-4eca-988e-073a4bd19f80\") " pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:43:46.409776 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:46.409759    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 21 17:43:46.417984 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:46.417969    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 21 17:43:46.429303 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:46.429275    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzhfd\" (UniqueName: \"kubernetes.io/projected/6db0adc0-a33d-4eca-988e-073a4bd19f80-kube-api-access-tzhfd\") pod \"network-check-target-brzgc\" (UID: \"6db0adc0-a33d-4eca-988e-073a4bd19f80\") " pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:43:46.724072 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:46.724046    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kt2hm\""
May 21 17:43:46.732166 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:46.732145    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:43:46.850991 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:46.850961    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-brzgc"]
May 21 17:43:46.854812 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:43:46.854770    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6db0adc0_a33d_4eca_988e_073a4bd19f80.slice/crio-c39c0a373c7aa55dc35a5d60199cb0c97ef945e78149378c894ae8ce92459caf WatchSource:0}: Error finding container c39c0a373c7aa55dc35a5d60199cb0c97ef945e78149378c894ae8ce92459caf: Status 404 returned error can't find the container with id c39c0a373c7aa55dc35a5d60199cb0c97ef945e78149378c894ae8ce92459caf
May 21 17:43:47.765831 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:47.765777    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-brzgc" event={"ID":"6db0adc0-a33d-4eca-988e-073a4bd19f80","Type":"ContainerStarted","Data":"c39c0a373c7aa55dc35a5d60199cb0c97ef945e78149378c894ae8ce92459caf"}
May 21 17:43:50.774948 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:50.774911    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-brzgc" event={"ID":"6db0adc0-a33d-4eca-988e-073a4bd19f80","Type":"ContainerStarted","Data":"0953a56cfa58728989a80fe108ad59f2647716137e6faf7512be7e816051e53e"}
May 21 17:43:50.775350 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:50.775125    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:43:50.788855 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:43:50.788812    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-brzgc" podStartSLOduration=67.033651524 podStartE2EDuration="1m10.788799639s" podCreationTimestamp="2026-05-21 17:42:40 +0000 UTC" firstStartedPulling="2026-05-21 17:43:46.857097349 +0000 UTC m=+66.967490307" lastFinishedPulling="2026-05-21 17:43:50.612245449 +0000 UTC m=+70.722638422" observedRunningTime="2026-05-21 17:43:50.787808196 +0000 UTC m=+70.898201173" watchObservedRunningTime="2026-05-21 17:43:50.788799639 +0000 UTC m=+70.899192613"
May 21 17:44:18.156639 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:44:18.156601    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-gwldh\" (UID: \"e505019e-ed7b-4564-861e-17d31ff967f5\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:44:18.157062 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:44:18.156651    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert\") pod \"ingress-canary-dd2vd\" (UID: \"ec348fe3-0beb-4db4-be82-ec7970b5c9e9\") " pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:44:18.157062 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:44:18.156686    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:44:18.157062 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:44:18.156721    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:44:18.157062 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:44:18.156738    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 21 17:44:18.157062 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:44:18.156831    2558 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 17:44:18.157062 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:44:18.156839    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 17:44:18.157062 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:44:18.156834    2558 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 21 17:44:18.157062 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:44:18.156881    2558 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f6c48bf8c-dlzc8: secret "image-registry-tls" not found
May 21 17:44:18.157062 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:44:18.156836    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert podName:e505019e-ed7b-4564-861e-17d31ff967f5 nodeName:}" failed. No retries permitted until 2026-05-21 17:45:22.156819397 +0000 UTC m=+162.267212361 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-gwldh" (UID: "e505019e-ed7b-4564-861e-17d31ff967f5") : secret "networking-console-plugin-cert" not found
May 21 17:44:18.157062 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:44:18.156919    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert podName:ec348fe3-0beb-4db4-be82-ec7970b5c9e9 nodeName:}" failed. No retries permitted until 2026-05-21 17:45:22.156906536 +0000 UTC m=+162.267299495 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert") pod "ingress-canary-dd2vd" (UID: "ec348fe3-0beb-4db4-be82-ec7970b5c9e9") : secret "canary-serving-cert" not found
May 21 17:44:18.157062 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:44:18.156929    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls podName:ceaa27ad-561f-481b-aa5d-59f9c8448199 nodeName:}" failed. No retries permitted until 2026-05-21 17:45:22.156923171 +0000 UTC m=+162.267316129 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls") pod "dns-default-phkwf" (UID: "ceaa27ad-561f-481b-aa5d-59f9c8448199") : secret "dns-default-metrics-tls" not found
May 21 17:44:18.157062 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:44:18.156940    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls podName:0a0046ef-374c-4543-9402-f60e026fcefa nodeName:}" failed. No retries permitted until 2026-05-21 17:45:22.156933534 +0000 UTC m=+162.267326491 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls") pod "image-registry-6f6c48bf8c-dlzc8" (UID: "0a0046ef-374c-4543-9402-f60e026fcefa") : secret "image-registry-tls" not found
May 21 17:44:21.782743 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:44:21.780013    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-brzgc"
May 21 17:44:50.284475 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:44:50.284435    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs\") pod \"network-metrics-daemon-sk2jf\" (UID: \"48a7b5cd-49ef-4177-83f1-80c264282bf9\") " pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:44:50.284900 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:44:50.284552    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 21 17:44:50.284900 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:44:50.284616    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs podName:48a7b5cd-49ef-4177-83f1-80c264282bf9 nodeName:}" failed. No retries permitted until 2026-05-21 17:46:52.284601016 +0000 UTC m=+252.394993978 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs") pod "network-metrics-daemon-sk2jf" (UID: "48a7b5cd-49ef-4177-83f1-80c264282bf9") : secret "metrics-daemon-secret" not found
May 21 17:45:17.240613 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:45:17.240564    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh" podUID="e505019e-ed7b-4564-861e-17d31ff967f5"
May 21 17:45:17.248759 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:45:17.248734    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8" podUID="0a0046ef-374c-4543-9402-f60e026fcefa"
May 21 17:45:17.256874 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:45:17.256849    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-phkwf" podUID="ceaa27ad-561f-481b-aa5d-59f9c8448199"
May 21 17:45:17.262972 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:45:17.262956    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-dd2vd" podUID="ec348fe3-0beb-4db4-be82-ec7970b5c9e9"
May 21 17:45:17.985201 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:17.985165    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:45:17.985390 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:17.985165    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:45:17.985390 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:17.985177    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-phkwf"
May 21 17:45:17.985515 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:17.985177    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:45:18.526568 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:45:18.526535    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-sk2jf" podUID="48a7b5cd-49ef-4177-83f1-80c264282bf9"
May 21 17:45:22.227573 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:22.227533    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls\") pod \"image-registry-6f6c48bf8c-dlzc8\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") " pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:45:22.227573 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:22.227576    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-gwldh\" (UID: \"e505019e-ed7b-4564-861e-17d31ff967f5\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:45:22.228027 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:22.227608    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert\") pod \"ingress-canary-dd2vd\" (UID: \"ec348fe3-0beb-4db4-be82-ec7970b5c9e9\") " pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:45:22.228027 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:22.227635    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:45:22.228027 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:45:22.227684    2558 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 21 17:45:22.228027 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:45:22.227704    2558 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f6c48bf8c-dlzc8: secret "image-registry-tls" not found
May 21 17:45:22.228027 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:45:22.227719    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 21 17:45:22.228027 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:45:22.227759    2558 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 21 17:45:22.228027 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:45:22.227759    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 21 17:45:22.228027 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:45:22.227770    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls podName:0a0046ef-374c-4543-9402-f60e026fcefa nodeName:}" failed. No retries permitted until 2026-05-21 17:47:24.227754015 +0000 UTC m=+284.338146972 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls") pod "image-registry-6f6c48bf8c-dlzc8" (UID: "0a0046ef-374c-4543-9402-f60e026fcefa") : secret "image-registry-tls" not found
May 21 17:45:22.228027 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:45:22.227859    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls podName:ceaa27ad-561f-481b-aa5d-59f9c8448199 nodeName:}" failed. No retries permitted until 2026-05-21 17:47:24.227847626 +0000 UTC m=+284.338240585 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls") pod "dns-default-phkwf" (UID: "ceaa27ad-561f-481b-aa5d-59f9c8448199") : secret "dns-default-metrics-tls" not found
May 21 17:45:22.228027 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:45:22.227869    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert podName:ec348fe3-0beb-4db4-be82-ec7970b5c9e9 nodeName:}" failed. No retries permitted until 2026-05-21 17:47:24.227863507 +0000 UTC m=+284.338256465 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert") pod "ingress-canary-dd2vd" (UID: "ec348fe3-0beb-4db4-be82-ec7970b5c9e9") : secret "canary-serving-cert" not found
May 21 17:45:22.228027 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:45:22.227877    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert podName:e505019e-ed7b-4564-861e-17d31ff967f5 nodeName:}" failed. No retries permitted until 2026-05-21 17:47:24.227872872 +0000 UTC m=+284.338265829 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-gwldh" (UID: "e505019e-ed7b-4564-861e-17d31ff967f5") : secret "networking-console-plugin-cert" not found
May 21 17:45:23.190647 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:23.190617    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-h9lc4_f95b9ee8-6015-4709-ac02-d7db0b48a238/dns-node-resolver/0.log"
May 21 17:45:24.395818 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:24.395772    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zcbrp_511e916d-ee83-4b83-a9e4-811ff6092f9a/node-ca/0.log"
May 21 17:45:24.703430 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:24.703379    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h" podUID="025463ef-f6b2-4ae5-9a5e-44831e3ede8c" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.12:8000/readyz\": dial tcp 10.132.0.12:8000: connect: connection refused"
May 21 17:45:25.001915 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:25.001833    2558 generic.go:358] "Generic (PLEG): container finished" podID="d469384a-983c-48c8-8d28-6f6ae5a22c4f" containerID="0c25ca6b63367e55014c3bce3a28a77de9dc83c99c3db5e5a5848ef02a4c7d14" exitCode=255
May 21 17:45:25.001915 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:25.001903    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86c4854848-gwvzm" event={"ID":"d469384a-983c-48c8-8d28-6f6ae5a22c4f","Type":"ContainerDied","Data":"0c25ca6b63367e55014c3bce3a28a77de9dc83c99c3db5e5a5848ef02a4c7d14"}
May 21 17:45:25.002268 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:25.002247    2558 scope.go:117] "RemoveContainer" containerID="0c25ca6b63367e55014c3bce3a28a77de9dc83c99c3db5e5a5848ef02a4c7d14"
May 21 17:45:25.003204 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:25.003184    2558 generic.go:358] "Generic (PLEG): container finished" podID="025463ef-f6b2-4ae5-9a5e-44831e3ede8c" containerID="736be79119ae6d72ad84f9acc81c61b97e1b06d6c88d23f08707ba7178fbf6b8" exitCode=1
May 21 17:45:25.003286 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:25.003220    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h" event={"ID":"025463ef-f6b2-4ae5-9a5e-44831e3ede8c","Type":"ContainerDied","Data":"736be79119ae6d72ad84f9acc81c61b97e1b06d6c88d23f08707ba7178fbf6b8"}
May 21 17:45:25.003522 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:25.003502    2558 scope.go:117] "RemoveContainer" containerID="736be79119ae6d72ad84f9acc81c61b97e1b06d6c88d23f08707ba7178fbf6b8"
May 21 17:45:26.007910 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:26.007872    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86c4854848-gwvzm" event={"ID":"d469384a-983c-48c8-8d28-6f6ae5a22c4f","Type":"ContainerStarted","Data":"9514f913417b11bd4776d9c737a1bb517c4a2761c06ed48927c7f69b7f23c412"}
May 21 17:45:26.009279 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:26.009259    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h" event={"ID":"025463ef-f6b2-4ae5-9a5e-44831e3ede8c","Type":"ContainerStarted","Data":"90f4a0039f4080731410f1ac240d585a948a7c7303cc6c93aace61e9c88b1b06"}
May 21 17:45:26.009507 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:26.009490    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"
May 21 17:45:26.010089 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:26.010072    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6985688c57-zqn5h"
May 21 17:45:33.502475 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:33.502391    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:45:44.702929 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.702895    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mwzc6"]
May 21 17:45:44.706030 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.706009    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:44.708347 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.708325    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 21 17:45:44.709044 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.709027    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 21 17:45:44.709116 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.709052    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9cn4m\""
May 21 17:45:44.709174 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.709054    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 21 17:45:44.710640 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.710624    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 21 17:45:44.716923 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.716905    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mwzc6"]
May 21 17:45:44.808808 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.808753    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8121325a-69bd-4adb-84ca-f10077e94863-crio-socket\") pod \"insights-runtime-extractor-mwzc6\" (UID: \"8121325a-69bd-4adb-84ca-f10077e94863\") " pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:44.808981 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.808813    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8121325a-69bd-4adb-84ca-f10077e94863-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwzc6\" (UID: \"8121325a-69bd-4adb-84ca-f10077e94863\") " pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:44.808981 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.808882    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v4l9\" (UniqueName: \"kubernetes.io/projected/8121325a-69bd-4adb-84ca-f10077e94863-kube-api-access-7v4l9\") pod \"insights-runtime-extractor-mwzc6\" (UID: \"8121325a-69bd-4adb-84ca-f10077e94863\") " pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:44.809066 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.808984    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8121325a-69bd-4adb-84ca-f10077e94863-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwzc6\" (UID: \"8121325a-69bd-4adb-84ca-f10077e94863\") " pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:44.809066 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.809022    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8121325a-69bd-4adb-84ca-f10077e94863-data-volume\") pod \"insights-runtime-extractor-mwzc6\" (UID: \"8121325a-69bd-4adb-84ca-f10077e94863\") " pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:44.910008 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.909966    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4l9\" (UniqueName: \"kubernetes.io/projected/8121325a-69bd-4adb-84ca-f10077e94863-kube-api-access-7v4l9\") pod \"insights-runtime-extractor-mwzc6\" (UID: \"8121325a-69bd-4adb-84ca-f10077e94863\") " pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:44.910187 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.910066    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8121325a-69bd-4adb-84ca-f10077e94863-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwzc6\" (UID: \"8121325a-69bd-4adb-84ca-f10077e94863\") " pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:44.910187 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.910094    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8121325a-69bd-4adb-84ca-f10077e94863-data-volume\") pod \"insights-runtime-extractor-mwzc6\" (UID: \"8121325a-69bd-4adb-84ca-f10077e94863\") " pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:44.910187 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.910132    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8121325a-69bd-4adb-84ca-f10077e94863-crio-socket\") pod \"insights-runtime-extractor-mwzc6\" (UID: \"8121325a-69bd-4adb-84ca-f10077e94863\") " pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:44.910323 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.910204    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8121325a-69bd-4adb-84ca-f10077e94863-crio-socket\") pod \"insights-runtime-extractor-mwzc6\" (UID: \"8121325a-69bd-4adb-84ca-f10077e94863\") " pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:44.910323 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.910250    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8121325a-69bd-4adb-84ca-f10077e94863-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwzc6\" (UID: \"8121325a-69bd-4adb-84ca-f10077e94863\") " pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:44.910513 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.910488    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8121325a-69bd-4adb-84ca-f10077e94863-data-volume\") pod \"insights-runtime-extractor-mwzc6\" (UID: \"8121325a-69bd-4adb-84ca-f10077e94863\") " pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:44.910739 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.910719    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8121325a-69bd-4adb-84ca-f10077e94863-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwzc6\" (UID: \"8121325a-69bd-4adb-84ca-f10077e94863\") " pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:44.912421 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.912405    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8121325a-69bd-4adb-84ca-f10077e94863-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwzc6\" (UID: \"8121325a-69bd-4adb-84ca-f10077e94863\") " pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:44.921946 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:44.921920    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v4l9\" (UniqueName: \"kubernetes.io/projected/8121325a-69bd-4adb-84ca-f10077e94863-kube-api-access-7v4l9\") pod \"insights-runtime-extractor-mwzc6\" (UID: \"8121325a-69bd-4adb-84ca-f10077e94863\") " pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:45.015044 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:45.014963    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mwzc6"
May 21 17:45:45.131098 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:45.131064    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mwzc6"]
May 21 17:45:45.134502 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:45:45.134478    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8121325a_69bd_4adb_84ca_f10077e94863.slice/crio-4fe3a10015ded9600f17640d49f67371c709e366a0365f9efcd7b33083836dae WatchSource:0}: Error finding container 4fe3a10015ded9600f17640d49f67371c709e366a0365f9efcd7b33083836dae: Status 404 returned error can't find the container with id 4fe3a10015ded9600f17640d49f67371c709e366a0365f9efcd7b33083836dae
May 21 17:45:46.059121 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:46.059093    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mwzc6" event={"ID":"8121325a-69bd-4adb-84ca-f10077e94863","Type":"ContainerStarted","Data":"e63ac768f79b6c99a36838c20a1099fe031491f0f7dcb921f42cc36d7aac3a14"}
May 21 17:45:46.059390 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:46.059135    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mwzc6" event={"ID":"8121325a-69bd-4adb-84ca-f10077e94863","Type":"ContainerStarted","Data":"531c6a30f4126950d0af79cab68268a02f4a4d6ac36408703d91b349f28de9a2"}
May 21 17:45:46.059390 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:46.059180    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mwzc6" event={"ID":"8121325a-69bd-4adb-84ca-f10077e94863","Type":"ContainerStarted","Data":"4fe3a10015ded9600f17640d49f67371c709e366a0365f9efcd7b33083836dae"}
May 21 17:45:46.287157 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:46.287075    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7" podUID="6cecd918-46ce-482d-8468-6b0157a1da42" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 21 17:45:48.067206 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:48.067164    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mwzc6" event={"ID":"8121325a-69bd-4adb-84ca-f10077e94863","Type":"ContainerStarted","Data":"e6fb013b04e209bc3b487de2a2f7b95de879517750287ad9c8ef6e761c16ccdc"}
May 21 17:45:48.083958 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:48.083915    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mwzc6" podStartSLOduration=1.426989519 podStartE2EDuration="4.083902664s" podCreationTimestamp="2026-05-21 17:45:44 +0000 UTC" firstStartedPulling="2026-05-21 17:45:45.18320984 +0000 UTC m=+185.293602797" lastFinishedPulling="2026-05-21 17:45:47.840122981 +0000 UTC m=+187.950515942" observedRunningTime="2026-05-21 17:45:48.082510885 +0000 UTC m=+188.192903865" watchObservedRunningTime="2026-05-21 17:45:48.083902664 +0000 UTC m=+188.194295643"
May 21 17:45:56.287207 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:56.287170    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7" podUID="6cecd918-46ce-482d-8468-6b0157a1da42" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 21 17:45:59.594429 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.594399    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b2wqm"]
May 21 17:45:59.597529 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.597512    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.599771 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.599746    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 21 17:45:59.599898 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.599875    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 21 17:45:59.600004 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.599987    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 21 17:45:59.600077 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.600009    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 21 17:45:59.600603 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.600587    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 21 17:45:59.600688 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.600624    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5mg5j\""
May 21 17:45:59.600744 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.600716    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 21 17:45:59.732632 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.732593    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scmhf\" (UniqueName: \"kubernetes.io/projected/417f466c-6a1b-4811-a948-424e72817c1f-kube-api-access-scmhf\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.732632 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.732633    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/417f466c-6a1b-4811-a948-424e72817c1f-sys\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.732854 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.732662    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/417f466c-6a1b-4811-a948-424e72817c1f-node-exporter-textfile\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.732854 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.732688    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/417f466c-6a1b-4811-a948-424e72817c1f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.732854 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.732724    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/417f466c-6a1b-4811-a948-424e72817c1f-root\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.732854 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.732740    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/417f466c-6a1b-4811-a948-424e72817c1f-node-exporter-accelerators-collector-config\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.732854 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.732806    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/417f466c-6a1b-4811-a948-424e72817c1f-metrics-client-ca\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.733006 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.732858    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/417f466c-6a1b-4811-a948-424e72817c1f-node-exporter-wtmp\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.733006 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.732889    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/417f466c-6a1b-4811-a948-424e72817c1f-node-exporter-tls\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.833591 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.833541    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/417f466c-6a1b-4811-a948-424e72817c1f-metrics-client-ca\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.833591 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.833604    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/417f466c-6a1b-4811-a948-424e72817c1f-node-exporter-wtmp\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.833813 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.833631    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/417f466c-6a1b-4811-a948-424e72817c1f-node-exporter-tls\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.833865 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.833808    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/417f466c-6a1b-4811-a948-424e72817c1f-node-exporter-wtmp\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.833865 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.833841    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-scmhf\" (UniqueName: \"kubernetes.io/projected/417f466c-6a1b-4811-a948-424e72817c1f-kube-api-access-scmhf\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.833947 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.833877    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/417f466c-6a1b-4811-a948-424e72817c1f-sys\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.833947 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.833905    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/417f466c-6a1b-4811-a948-424e72817c1f-node-exporter-textfile\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.833947 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.833921    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/417f466c-6a1b-4811-a948-424e72817c1f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.834088 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.833974    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/417f466c-6a1b-4811-a948-424e72817c1f-sys\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.834088 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.834050    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/417f466c-6a1b-4811-a948-424e72817c1f-root\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.834186 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.834082    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/417f466c-6a1b-4811-a948-424e72817c1f-node-exporter-accelerators-collector-config\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.834186 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.834161    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/417f466c-6a1b-4811-a948-424e72817c1f-root\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.834279 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.834223    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/417f466c-6a1b-4811-a948-424e72817c1f-node-exporter-textfile\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.834279 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.834249    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/417f466c-6a1b-4811-a948-424e72817c1f-metrics-client-ca\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.834577 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.834555    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/417f466c-6a1b-4811-a948-424e72817c1f-node-exporter-accelerators-collector-config\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.836051 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.836031    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/417f466c-6a1b-4811-a948-424e72817c1f-node-exporter-tls\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.836147 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.836063    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/417f466c-6a1b-4811-a948-424e72817c1f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.840964 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.840939    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scmhf\" (UniqueName: \"kubernetes.io/projected/417f466c-6a1b-4811-a948-424e72817c1f-kube-api-access-scmhf\") pod \"node-exporter-b2wqm\" (UID: \"417f466c-6a1b-4811-a948-424e72817c1f\") " pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.906797 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:45:59.906715    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b2wqm"
May 21 17:45:59.914675 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:45:59.914649    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod417f466c_6a1b_4811_a948_424e72817c1f.slice/crio-2f8e00a655acf4bcd1abac83a066e7baf5c826f5a5b91fdcbbf188d2f6d77a83 WatchSource:0}: Error finding container 2f8e00a655acf4bcd1abac83a066e7baf5c826f5a5b91fdcbbf188d2f6d77a83: Status 404 returned error can't find the container with id 2f8e00a655acf4bcd1abac83a066e7baf5c826f5a5b91fdcbbf188d2f6d77a83
May 21 17:46:00.095890 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:00.095852    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b2wqm" event={"ID":"417f466c-6a1b-4811-a948-424e72817c1f","Type":"ContainerStarted","Data":"2f8e00a655acf4bcd1abac83a066e7baf5c826f5a5b91fdcbbf188d2f6d77a83"}
May 21 17:46:01.099849 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:01.099768    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b2wqm" event={"ID":"417f466c-6a1b-4811-a948-424e72817c1f","Type":"ContainerStarted","Data":"42ec88a1baf4a22296b727f78d9625b2f630a781ddf404445b3f9bfaa2c10af0"}
May 21 17:46:02.103343 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:02.103308    2558 generic.go:358] "Generic (PLEG): container finished" podID="417f466c-6a1b-4811-a948-424e72817c1f" containerID="42ec88a1baf4a22296b727f78d9625b2f630a781ddf404445b3f9bfaa2c10af0" exitCode=0
May 21 17:46:02.103830 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:02.103355    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b2wqm" event={"ID":"417f466c-6a1b-4811-a948-424e72817c1f","Type":"ContainerDied","Data":"42ec88a1baf4a22296b727f78d9625b2f630a781ddf404445b3f9bfaa2c10af0"}
May 21 17:46:03.107893 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:03.107856    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b2wqm" event={"ID":"417f466c-6a1b-4811-a948-424e72817c1f","Type":"ContainerStarted","Data":"c6667f29450151c365fe0d07ee492e5e5251120eeee5b4e1dfc84196b3db190b"}
May 21 17:46:03.107893 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:03.107897    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b2wqm" event={"ID":"417f466c-6a1b-4811-a948-424e72817c1f","Type":"ContainerStarted","Data":"d2014f2f21096e1350bf0d042d7c8c95db3053183d1526e2a4f46fb68848dc59"}
May 21 17:46:03.129930 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:03.129892    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b2wqm" podStartSLOduration=3.00985392 podStartE2EDuration="4.129879284s" podCreationTimestamp="2026-05-21 17:45:59 +0000 UTC" firstStartedPulling="2026-05-21 17:45:59.916425572 +0000 UTC m=+200.026818546" lastFinishedPulling="2026-05-21 17:46:01.036450949 +0000 UTC m=+201.146843910" observedRunningTime="2026-05-21 17:46:03.129159404 +0000 UTC m=+203.239552386" watchObservedRunningTime="2026-05-21 17:46:03.129879284 +0000 UTC m=+203.240272313"
May 21 17:46:06.286837 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:06.286780    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7" podUID="6cecd918-46ce-482d-8468-6b0157a1da42" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 21 17:46:06.287199 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:06.286869    2558 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7"
May 21 17:46:06.287332 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:06.287302    2558 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"a7ee6ec9962151e58a129d14d5464c148b2dd230063bd3be244f093bdd85b4e3"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7" containerMessage="Container service-proxy failed liveness probe, will be restarted"
May 21 17:46:06.287371 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:06.287352    2558 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7" podUID="6cecd918-46ce-482d-8468-6b0157a1da42" containerName="service-proxy" containerID="cri-o://a7ee6ec9962151e58a129d14d5464c148b2dd230063bd3be244f093bdd85b4e3" gracePeriod=30
May 21 17:46:07.121669 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:07.121636    2558 generic.go:358] "Generic (PLEG): container finished" podID="6cecd918-46ce-482d-8468-6b0157a1da42" containerID="a7ee6ec9962151e58a129d14d5464c148b2dd230063bd3be244f093bdd85b4e3" exitCode=2
May 21 17:46:07.121851 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:07.121706    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7" event={"ID":"6cecd918-46ce-482d-8468-6b0157a1da42","Type":"ContainerDied","Data":"a7ee6ec9962151e58a129d14d5464c148b2dd230063bd3be244f093bdd85b4e3"}
May 21 17:46:07.121851 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:07.121741    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d565474b5-79sh7" event={"ID":"6cecd918-46ce-482d-8468-6b0157a1da42","Type":"ContainerStarted","Data":"537b2c8d239507a935c5c25d849bb29249a0470ea86b319014f8435df2f6072e"}
May 21 17:46:07.248844 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:07.248815    2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"]
May 21 17:46:07.249018 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:46:07.249000    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8" podUID="0a0046ef-374c-4543-9402-f60e026fcefa"
May 21 17:46:08.124216 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.124185    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:46:08.128151 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.128129    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:46:08.208352 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.208322    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a0046ef-374c-4543-9402-f60e026fcefa-ca-trust-extracted\") pod \"0a0046ef-374c-4543-9402-f60e026fcefa\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") "
May 21 17:46:08.208487 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.208360    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-bound-sa-token\") pod \"0a0046ef-374c-4543-9402-f60e026fcefa\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") "
May 21 17:46:08.208487 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.208413    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a0046ef-374c-4543-9402-f60e026fcefa-installation-pull-secrets\") pod \"0a0046ef-374c-4543-9402-f60e026fcefa\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") "
May 21 17:46:08.208487 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.208436    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a0046ef-374c-4543-9402-f60e026fcefa-trusted-ca\") pod \"0a0046ef-374c-4543-9402-f60e026fcefa\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") "
May 21 17:46:08.208487 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.208458    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0a0046ef-374c-4543-9402-f60e026fcefa-image-registry-private-configuration\") pod \"0a0046ef-374c-4543-9402-f60e026fcefa\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") "
May 21 17:46:08.208685 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.208499    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a0046ef-374c-4543-9402-f60e026fcefa-registry-certificates\") pod \"0a0046ef-374c-4543-9402-f60e026fcefa\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") "
May 21 17:46:08.208685 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.208525    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ccmp\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-kube-api-access-5ccmp\") pod \"0a0046ef-374c-4543-9402-f60e026fcefa\" (UID: \"0a0046ef-374c-4543-9402-f60e026fcefa\") "
May 21 17:46:08.208685 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.208634    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0046ef-374c-4543-9402-f60e026fcefa-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0a0046ef-374c-4543-9402-f60e026fcefa" (UID: "0a0046ef-374c-4543-9402-f60e026fcefa"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 21 17:46:08.208916 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.208888    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a0046ef-374c-4543-9402-f60e026fcefa-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0a0046ef-374c-4543-9402-f60e026fcefa" (UID: "0a0046ef-374c-4543-9402-f60e026fcefa"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 21 17:46:08.209088 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.208960    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a0046ef-374c-4543-9402-f60e026fcefa-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0a0046ef-374c-4543-9402-f60e026fcefa" (UID: "0a0046ef-374c-4543-9402-f60e026fcefa"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 21 17:46:08.209176 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.208948    2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a0046ef-374c-4543-9402-f60e026fcefa-ca-trust-extracted\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:46:08.210941 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.210907    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a0046ef-374c-4543-9402-f60e026fcefa-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0a0046ef-374c-4543-9402-f60e026fcefa" (UID: "0a0046ef-374c-4543-9402-f60e026fcefa"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 21 17:46:08.211035 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.210951    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0a0046ef-374c-4543-9402-f60e026fcefa" (UID: "0a0046ef-374c-4543-9402-f60e026fcefa"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 17:46:08.211035 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.210972    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a0046ef-374c-4543-9402-f60e026fcefa-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0a0046ef-374c-4543-9402-f60e026fcefa" (UID: "0a0046ef-374c-4543-9402-f60e026fcefa"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 21 17:46:08.211035 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.211010    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-kube-api-access-5ccmp" (OuterVolumeSpecName: "kube-api-access-5ccmp") pod "0a0046ef-374c-4543-9402-f60e026fcefa" (UID: "0a0046ef-374c-4543-9402-f60e026fcefa"). InnerVolumeSpecName "kube-api-access-5ccmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 17:46:08.309800 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.309758    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ccmp\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-kube-api-access-5ccmp\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:46:08.309800 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.309801    2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-bound-sa-token\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:46:08.309981 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.309812    2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a0046ef-374c-4543-9402-f60e026fcefa-installation-pull-secrets\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:46:08.309981 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.309823    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a0046ef-374c-4543-9402-f60e026fcefa-trusted-ca\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:46:08.309981 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.309832    2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0a0046ef-374c-4543-9402-f60e026fcefa-image-registry-private-configuration\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:46:08.309981 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:08.309841    2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a0046ef-374c-4543-9402-f60e026fcefa-registry-certificates\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:46:09.125967 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:09.125937    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"
May 21 17:46:09.157765 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:09.157729    2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"]
May 21 17:46:09.159362 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:09.159340    2558 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6f6c48bf8c-dlzc8"]
May 21 17:46:09.317340 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:09.317308    2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a0046ef-374c-4543-9402-f60e026fcefa-registry-tls\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:46:10.506998 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:10.506970    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a0046ef-374c-4543-9402-f60e026fcefa" path="/var/lib/kubelet/pods/0a0046ef-374c-4543-9402-f60e026fcefa/volumes"
May 21 17:46:52.353677 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:52.353630    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs\") pod \"network-metrics-daemon-sk2jf\" (UID: \"48a7b5cd-49ef-4177-83f1-80c264282bf9\") " pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:46:52.355862 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:52.355844    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48a7b5cd-49ef-4177-83f1-80c264282bf9-metrics-certs\") pod \"network-metrics-daemon-sk2jf\" (UID: \"48a7b5cd-49ef-4177-83f1-80c264282bf9\") " pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:46:52.405556 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:52.405515    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s4sh9\""
May 21 17:46:52.413402 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:52.413378    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk2jf"
May 21 17:46:52.531973 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:52.531945    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sk2jf"]
May 21 17:46:52.535180 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:46:52.535148    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48a7b5cd_49ef_4177_83f1_80c264282bf9.slice/crio-0a2136b512ccb02e53abbba81628700be35b91c5663887cc6d9c64edbb1af835 WatchSource:0}: Error finding container 0a2136b512ccb02e53abbba81628700be35b91c5663887cc6d9c64edbb1af835: Status 404 returned error can't find the container with id 0a2136b512ccb02e53abbba81628700be35b91c5663887cc6d9c64edbb1af835
May 21 17:46:53.229541 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:53.229509    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sk2jf" event={"ID":"48a7b5cd-49ef-4177-83f1-80c264282bf9","Type":"ContainerStarted","Data":"0a2136b512ccb02e53abbba81628700be35b91c5663887cc6d9c64edbb1af835"}
May 21 17:46:54.234177 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:54.234139    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sk2jf" event={"ID":"48a7b5cd-49ef-4177-83f1-80c264282bf9","Type":"ContainerStarted","Data":"9a8a16f93b42c65a6ab81df7abbfcd2240fd5c7661e6783d1f827079f5c09940"}
May 21 17:46:54.234177 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:54.234174    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sk2jf" event={"ID":"48a7b5cd-49ef-4177-83f1-80c264282bf9","Type":"ContainerStarted","Data":"7365df681f8e27108a893d3e3843c8109c3c5fddf1169b648504e09566efae82"}
May 21 17:46:54.250570 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:46:54.250506    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sk2jf" podStartSLOduration=252.902769561 podStartE2EDuration="4m14.250486999s" podCreationTimestamp="2026-05-21 17:42:40 +0000 UTC" firstStartedPulling="2026-05-21 17:46:52.537355082 +0000 UTC m=+252.647748041" lastFinishedPulling="2026-05-21 17:46:53.885072518 +0000 UTC m=+253.995465479" observedRunningTime="2026-05-21 17:46:54.249922299 +0000 UTC m=+254.360315280" watchObservedRunningTime="2026-05-21 17:46:54.250486999 +0000 UTC m=+254.360879980"
May 21 17:47:20.986278 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:47:20.986242    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-dd2vd" podUID="ec348fe3-0beb-4db4-be82-ec7970b5c9e9"
May 21 17:47:20.986688 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:47:20.986240    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-phkwf" podUID="ceaa27ad-561f-481b-aa5d-59f9c8448199"
May 21 17:47:20.986688 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:47:20.986250    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh" podUID="e505019e-ed7b-4564-861e-17d31ff967f5"
May 21 17:47:21.308843 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:21.308737    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:47:21.308843 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:21.308737    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:47:21.309032 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:21.308751    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-phkwf"
May 21 17:47:24.296961 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:24.296922    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-gwldh\" (UID: \"e505019e-ed7b-4564-861e-17d31ff967f5\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:47:24.296961 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:24.296967    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert\") pod \"ingress-canary-dd2vd\" (UID: \"ec348fe3-0beb-4db4-be82-ec7970b5c9e9\") " pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:47:24.297494 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:24.296989    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:47:24.299334 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:24.299307    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ceaa27ad-561f-481b-aa5d-59f9c8448199-metrics-tls\") pod \"dns-default-phkwf\" (UID: \"ceaa27ad-561f-481b-aa5d-59f9c8448199\") " pod="openshift-dns/dns-default-phkwf"
May 21 17:47:24.299492 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:24.299468    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e505019e-ed7b-4564-861e-17d31ff967f5-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-gwldh\" (UID: \"e505019e-ed7b-4564-861e-17d31ff967f5\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:47:24.299533 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:24.299502    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec348fe3-0beb-4db4-be82-ec7970b5c9e9-cert\") pod \"ingress-canary-dd2vd\" (UID: \"ec348fe3-0beb-4db4-be82-ec7970b5c9e9\") " pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:47:24.312302 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:24.312281    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wq5rz\""
May 21 17:47:24.312431 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:24.312281    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-r7lth\""
May 21 17:47:24.312431 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:24.312287    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5t9n8\""
May 21 17:47:24.319482 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:24.319464    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"
May 21 17:47:24.319482 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:24.319474    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dd2vd"
May 21 17:47:24.319635 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:24.319495    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-phkwf"
May 21 17:47:24.497169 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:24.497138    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dd2vd"]
May 21 17:47:24.500823 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:47:24.500778    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec348fe3_0beb_4db4_be82_ec7970b5c9e9.slice/crio-a2f2d777adea0a1ed3536ddf9ec2a18a10a07897e9ed088cb84742371c4ad8fc WatchSource:0}: Error finding container a2f2d777adea0a1ed3536ddf9ec2a18a10a07897e9ed088cb84742371c4ad8fc: Status 404 returned error can't find the container with id a2f2d777adea0a1ed3536ddf9ec2a18a10a07897e9ed088cb84742371c4ad8fc
May 21 17:47:24.669426 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:24.669393    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh"]
May 21 17:47:24.672564 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:24.672541    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-phkwf"]
May 21 17:47:24.673636 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:47:24.673532    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode505019e_ed7b_4564_861e_17d31ff967f5.slice/crio-74220b9145b04fa419f15bf7b16e01201230518cb18871f7735ac6439e140f2f WatchSource:0}: Error finding container 74220b9145b04fa419f15bf7b16e01201230518cb18871f7735ac6439e140f2f: Status 404 returned error can't find the container with id 74220b9145b04fa419f15bf7b16e01201230518cb18871f7735ac6439e140f2f
May 21 17:47:24.675545 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:47:24.675525    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceaa27ad_561f_481b_aa5d_59f9c8448199.slice/crio-33702c81a86a292f8437b0872b30e6e86ef262b6899fa385a3b878976c037f91 WatchSource:0}: Error finding container 33702c81a86a292f8437b0872b30e6e86ef262b6899fa385a3b878976c037f91: Status 404 returned error can't find the container with id 33702c81a86a292f8437b0872b30e6e86ef262b6899fa385a3b878976c037f91
May 21 17:47:25.321651 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:25.321559    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dd2vd" event={"ID":"ec348fe3-0beb-4db4-be82-ec7970b5c9e9","Type":"ContainerStarted","Data":"a2f2d777adea0a1ed3536ddf9ec2a18a10a07897e9ed088cb84742371c4ad8fc"}
May 21 17:47:25.323170 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:25.323130    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-phkwf" event={"ID":"ceaa27ad-561f-481b-aa5d-59f9c8448199","Type":"ContainerStarted","Data":"33702c81a86a292f8437b0872b30e6e86ef262b6899fa385a3b878976c037f91"}
May 21 17:47:25.324337 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:25.324313    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh" event={"ID":"e505019e-ed7b-4564-861e-17d31ff967f5","Type":"ContainerStarted","Data":"74220b9145b04fa419f15bf7b16e01201230518cb18871f7735ac6439e140f2f"}
May 21 17:47:27.331568 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:27.331518    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh" event={"ID":"e505019e-ed7b-4564-861e-17d31ff967f5","Type":"ContainerStarted","Data":"0eb16bfaa584e3586af662b169788461b4969b5834e770106b4d6455ec388734"}
May 21 17:47:27.332876 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:27.332849    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dd2vd" event={"ID":"ec348fe3-0beb-4db4-be82-ec7970b5c9e9","Type":"ContainerStarted","Data":"6961f293a8ca30a538058a1749f7b88eb3dfe02b149b9e206b7e5b4eb644e0bb"}
May 21 17:47:27.334281 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:27.334257    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-phkwf" event={"ID":"ceaa27ad-561f-481b-aa5d-59f9c8448199","Type":"ContainerStarted","Data":"09b1d811e01cde9539e892433e2249b709a21cfa79279e3d40be6334aa169ba8"}
May 21 17:47:27.334281 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:27.334283    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-phkwf" event={"ID":"ceaa27ad-561f-481b-aa5d-59f9c8448199","Type":"ContainerStarted","Data":"677b58ddcbf0c6ad3b919a94b921db9ce0499c8c3e4f3e96bf264e322e6c83d5"}
May 21 17:47:27.334449 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:27.334377    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-phkwf"
May 21 17:47:27.347644 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:27.347604    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-gwldh" podStartSLOduration=266.099934386 podStartE2EDuration="4m28.347591385s" podCreationTimestamp="2026-05-21 17:42:59 +0000 UTC" firstStartedPulling="2026-05-21 17:47:24.67550811 +0000 UTC m=+284.785901068" lastFinishedPulling="2026-05-21 17:47:26.92316511 +0000 UTC m=+287.033558067" observedRunningTime="2026-05-21 17:47:27.34561468 +0000 UTC m=+287.456007659" watchObservedRunningTime="2026-05-21 17:47:27.347591385 +0000 UTC m=+287.457984365"
May 21 17:47:27.360477 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:27.360426    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dd2vd" podStartSLOduration=250.93836193 podStartE2EDuration="4m13.360407519s" podCreationTimestamp="2026-05-21 17:43:14 +0000 UTC" firstStartedPulling="2026-05-21 17:47:24.506593424 +0000 UTC m=+284.616986388" lastFinishedPulling="2026-05-21 17:47:26.928639005 +0000 UTC m=+287.039031977" observedRunningTime="2026-05-21 17:47:27.359189472 +0000 UTC m=+287.469582453" watchObservedRunningTime="2026-05-21 17:47:27.360407519 +0000 UTC m=+287.470800501"
May 21 17:47:27.375298 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:27.375261    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-phkwf" podStartSLOduration=251.127750608 podStartE2EDuration="4m13.375247702s" podCreationTimestamp="2026-05-21 17:43:14 +0000 UTC" firstStartedPulling="2026-05-21 17:47:24.677274067 +0000 UTC m=+284.787667025" lastFinishedPulling="2026-05-21 17:47:26.92477116 +0000 UTC m=+287.035164119" observedRunningTime="2026-05-21 17:47:27.37413675 +0000 UTC m=+287.484529730" watchObservedRunningTime="2026-05-21 17:47:27.375247702 +0000 UTC m=+287.485640682"
May 21 17:47:37.338742 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:37.338711    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-phkwf"
May 21 17:47:40.392806 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:40.392756    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 17:47:40.393262 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:40.393032    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 17:47:40.397307 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:47:40.397284    2558 kubelet.go:1628] "Image garbage collection succeeded"
May 21 17:50:14.845070 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:14.845037    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-z4l5x"]
May 21 17:50:14.847929 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:14.847913    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-z4l5x"
May 21 17:50:14.850114 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:14.850094    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
May 21 17:50:14.850295 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:14.850280    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
May 21 17:50:14.850844 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:14.850830    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-8tctk\""
May 21 17:50:14.854633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:14.854610    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-z4l5x"]
May 21 17:50:14.984382 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:14.984351    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c30d380-ec03-4e47-82e4-95baca65c3e8-bound-sa-token\") pod \"cert-manager-79c8d999ff-z4l5x\" (UID: \"3c30d380-ec03-4e47-82e4-95baca65c3e8\") " pod="cert-manager/cert-manager-79c8d999ff-z4l5x"
May 21 17:50:14.984559 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:14.984392    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czgch\" (UniqueName: \"kubernetes.io/projected/3c30d380-ec03-4e47-82e4-95baca65c3e8-kube-api-access-czgch\") pod \"cert-manager-79c8d999ff-z4l5x\" (UID: \"3c30d380-ec03-4e47-82e4-95baca65c3e8\") " pod="cert-manager/cert-manager-79c8d999ff-z4l5x"
May 21 17:50:15.085585 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:15.085551    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-czgch\" (UniqueName: \"kubernetes.io/projected/3c30d380-ec03-4e47-82e4-95baca65c3e8-kube-api-access-czgch\") pod \"cert-manager-79c8d999ff-z4l5x\" (UID: \"3c30d380-ec03-4e47-82e4-95baca65c3e8\") " pod="cert-manager/cert-manager-79c8d999ff-z4l5x"
May 21 17:50:15.085742 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:15.085637    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c30d380-ec03-4e47-82e4-95baca65c3e8-bound-sa-token\") pod \"cert-manager-79c8d999ff-z4l5x\" (UID: \"3c30d380-ec03-4e47-82e4-95baca65c3e8\") " pod="cert-manager/cert-manager-79c8d999ff-z4l5x"
May 21 17:50:15.093151 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:15.093119    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c30d380-ec03-4e47-82e4-95baca65c3e8-bound-sa-token\") pod \"cert-manager-79c8d999ff-z4l5x\" (UID: \"3c30d380-ec03-4e47-82e4-95baca65c3e8\") " pod="cert-manager/cert-manager-79c8d999ff-z4l5x"
May 21 17:50:15.093344 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:15.093321    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czgch\" (UniqueName: \"kubernetes.io/projected/3c30d380-ec03-4e47-82e4-95baca65c3e8-kube-api-access-czgch\") pod \"cert-manager-79c8d999ff-z4l5x\" (UID: \"3c30d380-ec03-4e47-82e4-95baca65c3e8\") " pod="cert-manager/cert-manager-79c8d999ff-z4l5x"
May 21 17:50:15.157212 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:15.157133    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-z4l5x"
May 21 17:50:15.282228 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:15.282195    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-z4l5x"]
May 21 17:50:15.285540 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:50:15.285511    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c30d380_ec03_4e47_82e4_95baca65c3e8.slice/crio-faeeb1636ba1182c63a1aebd2c91d25dbbc3d119e2b2aeb61a6a8eda3b5f33a8 WatchSource:0}: Error finding container faeeb1636ba1182c63a1aebd2c91d25dbbc3d119e2b2aeb61a6a8eda3b5f33a8: Status 404 returned error can't find the container with id faeeb1636ba1182c63a1aebd2c91d25dbbc3d119e2b2aeb61a6a8eda3b5f33a8
May 21 17:50:15.287743 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:15.287731    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 21 17:50:15.761396 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:15.761357    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-z4l5x" event={"ID":"3c30d380-ec03-4e47-82e4-95baca65c3e8","Type":"ContainerStarted","Data":"faeeb1636ba1182c63a1aebd2c91d25dbbc3d119e2b2aeb61a6a8eda3b5f33a8"}
May 21 17:50:18.771341 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:18.771303    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-z4l5x" event={"ID":"3c30d380-ec03-4e47-82e4-95baca65c3e8","Type":"ContainerStarted","Data":"441808c709ead9f3c52c1890858376a621b03c55ee4d112026210bf50fb897b3"}
May 21 17:50:18.788310 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:18.788266    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-z4l5x" podStartSLOduration=2.034509366 podStartE2EDuration="4.788252141s" podCreationTimestamp="2026-05-21 17:50:14 +0000 UTC" firstStartedPulling="2026-05-21 17:50:15.287869268 +0000 UTC m=+455.398262225" lastFinishedPulling="2026-05-21 17:50:18.041612039 +0000 UTC m=+458.152005000" observedRunningTime="2026-05-21 17:50:18.786522007 +0000 UTC m=+458.896914986" watchObservedRunningTime="2026-05-21 17:50:18.788252141 +0000 UTC m=+458.898645121"
May 21 17:50:31.928727 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:31.928689    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4"]
May 21 17:50:31.932547 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:31.932516    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4"
May 21 17:50:31.935093 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:31.935068    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
May 21 17:50:31.935233 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:31.935185    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
May 21 17:50:31.935233 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:31.935218    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6hwvz\""
May 21 17:50:31.935358 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:31.935278    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
May 21 17:50:31.935560 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:31.935545    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
May 21 17:50:31.943924 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:31.943902    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4"]
May 21 17:50:32.107118 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:32.107082    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shhxk\" (UniqueName: \"kubernetes.io/projected/2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6-kube-api-access-shhxk\") pod \"opendatahub-operator-controller-manager-5b676496df-mhnf4\" (UID: \"2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6\") " pod="opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4"
May 21 17:50:32.107287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:32.107130    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b676496df-mhnf4\" (UID: \"2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6\") " pod="opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4"
May 21 17:50:32.107287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:32.107204    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b676496df-mhnf4\" (UID: \"2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6\") " pod="opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4"
May 21 17:50:32.208043 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:32.207964    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b676496df-mhnf4\" (UID: \"2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6\") " pod="opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4"
May 21 17:50:32.208043 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:32.208011    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b676496df-mhnf4\" (UID: \"2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6\") " pod="opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4"
May 21 17:50:32.208185 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:32.208057    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-shhxk\" (UniqueName: \"kubernetes.io/projected/2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6-kube-api-access-shhxk\") pod \"opendatahub-operator-controller-manager-5b676496df-mhnf4\" (UID: \"2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6\") " pod="opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4"
May 21 17:50:32.210558 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:32.210530    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b676496df-mhnf4\" (UID: \"2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6\") " pod="opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4"
May 21 17:50:32.210671 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:32.210567    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b676496df-mhnf4\" (UID: \"2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6\") " pod="opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4"
May 21 17:50:32.214906 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:32.214887    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shhxk\" (UniqueName: \"kubernetes.io/projected/2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6-kube-api-access-shhxk\") pod \"opendatahub-operator-controller-manager-5b676496df-mhnf4\" (UID: \"2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6\") " pod="opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4"
May 21 17:50:32.243341 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:32.243314    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4"
May 21 17:50:32.369842 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:32.369810    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4"]
May 21 17:50:32.373532 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:50:32.373506    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d50c4a3_7bed_40fe_b6a8_f1f64bc907e6.slice/crio-424f4ee848aa359b43d8608dc0c22ef82f78144c131373836d3a2181191a7e42 WatchSource:0}: Error finding container 424f4ee848aa359b43d8608dc0c22ef82f78144c131373836d3a2181191a7e42: Status 404 returned error can't find the container with id 424f4ee848aa359b43d8608dc0c22ef82f78144c131373836d3a2181191a7e42
May 21 17:50:32.810420 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:32.810383    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4" event={"ID":"2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6","Type":"ContainerStarted","Data":"424f4ee848aa359b43d8608dc0c22ef82f78144c131373836d3a2181191a7e42"}
May 21 17:50:35.819416 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:35.819377    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4" event={"ID":"2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6","Type":"ContainerStarted","Data":"85f9bcf71241fff898bfb81c51dc759e1e60564c759f611e6b7b8d6b59f2af25"}
May 21 17:50:35.819879 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:35.819485    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4"
May 21 17:50:35.839881 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:35.839833    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4" podStartSLOduration=2.34709841 podStartE2EDuration="4.839819038s" podCreationTimestamp="2026-05-21 17:50:31 +0000 UTC" firstStartedPulling="2026-05-21 17:50:32.375175903 +0000 UTC m=+472.485568860" lastFinishedPulling="2026-05-21 17:50:34.867896526 +0000 UTC m=+474.978289488" observedRunningTime="2026-05-21 17:50:35.839196479 +0000 UTC m=+475.949589465" watchObservedRunningTime="2026-05-21 17:50:35.839819038 +0000 UTC m=+475.950212017"
May 21 17:50:46.824726 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:46.824693    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5b676496df-mhnf4"
May 21 17:50:52.472530 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:52.472494    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-nh67r"]
May 21 17:50:52.546952 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:52.546919    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-nh67r"
May 21 17:50:52.549047 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:52.549009    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
May 21 17:50:52.549047 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:52.549029    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-4n42b\""
May 21 17:50:52.551270 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:52.551251    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-nh67r"]
May 21 17:50:52.646954 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:52.646920    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2c025d5-a928-437c-8a6a-26f8be22402b-cert\") pod \"odh-model-controller-858dbf95b8-nh67r\" (UID: \"e2c025d5-a928-437c-8a6a-26f8be22402b\") " pod="opendatahub/odh-model-controller-858dbf95b8-nh67r"
May 21 17:50:52.647119 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:52.646970    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j8bw\" (UniqueName: \"kubernetes.io/projected/e2c025d5-a928-437c-8a6a-26f8be22402b-kube-api-access-7j8bw\") pod \"odh-model-controller-858dbf95b8-nh67r\" (UID: \"e2c025d5-a928-437c-8a6a-26f8be22402b\") " pod="opendatahub/odh-model-controller-858dbf95b8-nh67r"
May 21 17:50:52.747534 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:52.747440    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2c025d5-a928-437c-8a6a-26f8be22402b-cert\") pod \"odh-model-controller-858dbf95b8-nh67r\" (UID: \"e2c025d5-a928-437c-8a6a-26f8be22402b\") " pod="opendatahub/odh-model-controller-858dbf95b8-nh67r"
May 21 17:50:52.747534 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:52.747491    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7j8bw\" (UniqueName: \"kubernetes.io/projected/e2c025d5-a928-437c-8a6a-26f8be22402b-kube-api-access-7j8bw\") pod \"odh-model-controller-858dbf95b8-nh67r\" (UID: \"e2c025d5-a928-437c-8a6a-26f8be22402b\") " pod="opendatahub/odh-model-controller-858dbf95b8-nh67r"
May 21 17:50:52.747698 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:50:52.747578    2558 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
May 21 17:50:52.747698 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:50:52.747649    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2c025d5-a928-437c-8a6a-26f8be22402b-cert podName:e2c025d5-a928-437c-8a6a-26f8be22402b nodeName:}" failed. No retries permitted until 2026-05-21 17:50:53.247631752 +0000 UTC m=+493.358024709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e2c025d5-a928-437c-8a6a-26f8be22402b-cert") pod "odh-model-controller-858dbf95b8-nh67r" (UID: "e2c025d5-a928-437c-8a6a-26f8be22402b") : secret "odh-model-controller-webhook-cert" not found
May 21 17:50:52.758608 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:52.758579    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j8bw\" (UniqueName: \"kubernetes.io/projected/e2c025d5-a928-437c-8a6a-26f8be22402b-kube-api-access-7j8bw\") pod \"odh-model-controller-858dbf95b8-nh67r\" (UID: \"e2c025d5-a928-437c-8a6a-26f8be22402b\") " pod="opendatahub/odh-model-controller-858dbf95b8-nh67r"
May 21 17:50:53.251369 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:53.251331    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2c025d5-a928-437c-8a6a-26f8be22402b-cert\") pod \"odh-model-controller-858dbf95b8-nh67r\" (UID: \"e2c025d5-a928-437c-8a6a-26f8be22402b\") " pod="opendatahub/odh-model-controller-858dbf95b8-nh67r"
May 21 17:50:53.253729 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:53.253703    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2c025d5-a928-437c-8a6a-26f8be22402b-cert\") pod \"odh-model-controller-858dbf95b8-nh67r\" (UID: \"e2c025d5-a928-437c-8a6a-26f8be22402b\") " pod="opendatahub/odh-model-controller-858dbf95b8-nh67r"
May 21 17:50:53.456861 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:53.456807    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-nh67r"
May 21 17:50:53.574485 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:53.574449    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-nh67r"]
May 21 17:50:53.577686 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:50:53.577659    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2c025d5_a928_437c_8a6a_26f8be22402b.slice/crio-ea0c40aed4cbc9731c6789dba716e10cfb4269e1186c8c53654cb0d1aee1859f WatchSource:0}: Error finding container ea0c40aed4cbc9731c6789dba716e10cfb4269e1186c8c53654cb0d1aee1859f: Status 404 returned error can't find the container with id ea0c40aed4cbc9731c6789dba716e10cfb4269e1186c8c53654cb0d1aee1859f
May 21 17:50:53.869523 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:53.869442    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-nh67r" event={"ID":"e2c025d5-a928-437c-8a6a-26f8be22402b","Type":"ContainerStarted","Data":"ea0c40aed4cbc9731c6789dba716e10cfb4269e1186c8c53654cb0d1aee1859f"}
May 21 17:50:56.878212 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:56.878178    2558 generic.go:358] "Generic (PLEG): container finished" podID="e2c025d5-a928-437c-8a6a-26f8be22402b" containerID="949d9602911b92013d719079a245aefbda5486e85edf1e371d3e983531347400" exitCode=1
May 21 17:50:56.878668 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:56.878266    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-nh67r" event={"ID":"e2c025d5-a928-437c-8a6a-26f8be22402b","Type":"ContainerDied","Data":"949d9602911b92013d719079a245aefbda5486e85edf1e371d3e983531347400"}
May 21 17:50:56.878668 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:56.878442    2558 scope.go:117] "RemoveContainer" containerID="949d9602911b92013d719079a245aefbda5486e85edf1e371d3e983531347400"
May 21 17:50:57.171583 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:57.171562    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-nh5d7"]
May 21 17:50:57.174353 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:57.174333    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-nh5d7"
May 21 17:50:57.176624 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:57.176607    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
May 21 17:50:57.176732 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:57.176717    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-6hkps\""
May 21 17:50:57.178191 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:57.178168    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xrls\" (UniqueName: \"kubernetes.io/projected/f213e0dc-1d4d-46d6-a740-e1f44d9c2471-kube-api-access-8xrls\") pod \"kserve-controller-manager-856948b99f-nh5d7\" (UID: \"f213e0dc-1d4d-46d6-a740-e1f44d9c2471\") " pod="opendatahub/kserve-controller-manager-856948b99f-nh5d7"
May 21 17:50:57.178264 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:57.178226    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f213e0dc-1d4d-46d6-a740-e1f44d9c2471-cert\") pod \"kserve-controller-manager-856948b99f-nh5d7\" (UID: \"f213e0dc-1d4d-46d6-a740-e1f44d9c2471\") " pod="opendatahub/kserve-controller-manager-856948b99f-nh5d7"
May 21 17:50:57.187117 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:57.187091    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-nh5d7"]
May 21 17:50:57.279357 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:57.279324    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8xrls\" (UniqueName: \"kubernetes.io/projected/f213e0dc-1d4d-46d6-a740-e1f44d9c2471-kube-api-access-8xrls\") pod \"kserve-controller-manager-856948b99f-nh5d7\" (UID: \"f213e0dc-1d4d-46d6-a740-e1f44d9c2471\") " pod="opendatahub/kserve-controller-manager-856948b99f-nh5d7"
May 21 17:50:57.279517 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:57.279366    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f213e0dc-1d4d-46d6-a740-e1f44d9c2471-cert\") pod \"kserve-controller-manager-856948b99f-nh5d7\" (UID: \"f213e0dc-1d4d-46d6-a740-e1f44d9c2471\") " pod="opendatahub/kserve-controller-manager-856948b99f-nh5d7"
May 21 17:50:57.279517 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:50:57.279480    2558 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
May 21 17:50:57.279591 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:50:57.279535    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f213e0dc-1d4d-46d6-a740-e1f44d9c2471-cert podName:f213e0dc-1d4d-46d6-a740-e1f44d9c2471 nodeName:}" failed. No retries permitted until 2026-05-21 17:50:57.779516782 +0000 UTC m=+497.889909741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f213e0dc-1d4d-46d6-a740-e1f44d9c2471-cert") pod "kserve-controller-manager-856948b99f-nh5d7" (UID: "f213e0dc-1d4d-46d6-a740-e1f44d9c2471") : secret "kserve-webhook-server-cert" not found
May 21 17:50:57.290612 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:57.290584    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xrls\" (UniqueName: \"kubernetes.io/projected/f213e0dc-1d4d-46d6-a740-e1f44d9c2471-kube-api-access-8xrls\") pod \"kserve-controller-manager-856948b99f-nh5d7\" (UID: \"f213e0dc-1d4d-46d6-a740-e1f44d9c2471\") " pod="opendatahub/kserve-controller-manager-856948b99f-nh5d7"
May 21 17:50:57.784925 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:57.784882    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f213e0dc-1d4d-46d6-a740-e1f44d9c2471-cert\") pod \"kserve-controller-manager-856948b99f-nh5d7\" (UID: \"f213e0dc-1d4d-46d6-a740-e1f44d9c2471\") " pod="opendatahub/kserve-controller-manager-856948b99f-nh5d7"
May 21 17:50:57.785125 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:50:57.785039    2558 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
May 21 17:50:57.785125 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:50:57.785105    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f213e0dc-1d4d-46d6-a740-e1f44d9c2471-cert podName:f213e0dc-1d4d-46d6-a740-e1f44d9c2471 nodeName:}" failed. No retries permitted until 2026-05-21 17:50:58.785087915 +0000 UTC m=+498.895480874 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f213e0dc-1d4d-46d6-a740-e1f44d9c2471-cert") pod "kserve-controller-manager-856948b99f-nh5d7" (UID: "f213e0dc-1d4d-46d6-a740-e1f44d9c2471") : secret "kserve-webhook-server-cert" not found
May 21 17:50:57.884393 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:57.884363    2558 generic.go:358] "Generic (PLEG): container finished" podID="e2c025d5-a928-437c-8a6a-26f8be22402b" containerID="3421a51dbede6e94397c0da12d6b888ba87b393e3fee969a54872db846c792b9" exitCode=1
May 21 17:50:57.884823 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:57.884408    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-nh67r" event={"ID":"e2c025d5-a928-437c-8a6a-26f8be22402b","Type":"ContainerDied","Data":"3421a51dbede6e94397c0da12d6b888ba87b393e3fee969a54872db846c792b9"}
May 21 17:50:57.884823 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:57.884436    2558 scope.go:117] "RemoveContainer" containerID="949d9602911b92013d719079a245aefbda5486e85edf1e371d3e983531347400"
May 21 17:50:57.884823 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:57.884636    2558 scope.go:117] "RemoveContainer" containerID="3421a51dbede6e94397c0da12d6b888ba87b393e3fee969a54872db846c792b9"
May 21 17:50:57.884972 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:50:57.884833    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-nh67r_opendatahub(e2c025d5-a928-437c-8a6a-26f8be22402b)\"" pod="opendatahub/odh-model-controller-858dbf95b8-nh67r" podUID="e2c025d5-a928-437c-8a6a-26f8be22402b"
May 21 17:50:58.791626 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:58.791590    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f213e0dc-1d4d-46d6-a740-e1f44d9c2471-cert\") pod \"kserve-controller-manager-856948b99f-nh5d7\" (UID: \"f213e0dc-1d4d-46d6-a740-e1f44d9c2471\") " pod="opendatahub/kserve-controller-manager-856948b99f-nh5d7"
May 21 17:50:58.794022 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:58.793999    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f213e0dc-1d4d-46d6-a740-e1f44d9c2471-cert\") pod \"kserve-controller-manager-856948b99f-nh5d7\" (UID: \"f213e0dc-1d4d-46d6-a740-e1f44d9c2471\") " pod="opendatahub/kserve-controller-manager-856948b99f-nh5d7"
May 21 17:50:58.888186 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:58.888156    2558 scope.go:117] "RemoveContainer" containerID="3421a51dbede6e94397c0da12d6b888ba87b393e3fee969a54872db846c792b9"
May 21 17:50:58.888536 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:50:58.888339    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-nh67r_opendatahub(e2c025d5-a928-437c-8a6a-26f8be22402b)\"" pod="opendatahub/odh-model-controller-858dbf95b8-nh67r" podUID="e2c025d5-a928-437c-8a6a-26f8be22402b"
May 21 17:50:59.004659 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:59.004620    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-nh5d7"
May 21 17:50:59.138056 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:59.138027    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-nh5d7"]
May 21 17:50:59.140597 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:50:59.140570    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf213e0dc_1d4d_46d6_a740_e1f44d9c2471.slice/crio-e9eb44d66dc96609cdc39a76f64b2222b9f10a59ce9e941845d83d4c5cd44df0 WatchSource:0}: Error finding container e9eb44d66dc96609cdc39a76f64b2222b9f10a59ce9e941845d83d4c5cd44df0: Status 404 returned error can't find the container with id e9eb44d66dc96609cdc39a76f64b2222b9f10a59ce9e941845d83d4c5cd44df0
May 21 17:50:59.891148 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:50:59.891107    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-nh5d7" event={"ID":"f213e0dc-1d4d-46d6-a740-e1f44d9c2471","Type":"ContainerStarted","Data":"e9eb44d66dc96609cdc39a76f64b2222b9f10a59ce9e941845d83d4c5cd44df0"}
May 21 17:51:00.017076 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.017043    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc"]
May 21 17:51:00.020086 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.020070    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc"
May 21 17:51:00.023555 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.023528    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
May 21 17:51:00.023684 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.023669    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
May 21 17:51:00.024073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.024055    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
May 21 17:51:00.024263 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.024250    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
May 21 17:51:00.024982 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.024968    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-x2sct\""
May 21 17:51:00.033458 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.033439    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc"]
May 21 17:51:00.101628 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.101598    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b13d3629-00bd-42cf-b91e-b713b014618e-tmp\") pod \"kube-auth-proxy-54f55555d9-tkwmc\" (UID: \"b13d3629-00bd-42cf-b91e-b713b014618e\") " pod="openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc"
May 21 17:51:00.101828 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.101634    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b13d3629-00bd-42cf-b91e-b713b014618e-tls-certs\") pod \"kube-auth-proxy-54f55555d9-tkwmc\" (UID: \"b13d3629-00bd-42cf-b91e-b713b014618e\") " pod="openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc"
May 21 17:51:00.101828 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.101733    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrgzw\" (UniqueName: \"kubernetes.io/projected/b13d3629-00bd-42cf-b91e-b713b014618e-kube-api-access-qrgzw\") pod \"kube-auth-proxy-54f55555d9-tkwmc\" (UID: \"b13d3629-00bd-42cf-b91e-b713b014618e\") " pod="openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc"
May 21 17:51:00.202183 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.202141    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b13d3629-00bd-42cf-b91e-b713b014618e-tmp\") pod \"kube-auth-proxy-54f55555d9-tkwmc\" (UID: \"b13d3629-00bd-42cf-b91e-b713b014618e\") " pod="openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc"
May 21 17:51:00.202374 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.202200    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b13d3629-00bd-42cf-b91e-b713b014618e-tls-certs\") pod \"kube-auth-proxy-54f55555d9-tkwmc\" (UID: \"b13d3629-00bd-42cf-b91e-b713b014618e\") " pod="openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc"
May 21 17:51:00.202374 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.202263    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qrgzw\" (UniqueName: \"kubernetes.io/projected/b13d3629-00bd-42cf-b91e-b713b014618e-kube-api-access-qrgzw\") pod \"kube-auth-proxy-54f55555d9-tkwmc\" (UID: \"b13d3629-00bd-42cf-b91e-b713b014618e\") " pod="openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc"
May 21 17:51:00.202533 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:51:00.202415    2558 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
May 21 17:51:00.202533 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:51:00.202493    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b13d3629-00bd-42cf-b91e-b713b014618e-tls-certs podName:b13d3629-00bd-42cf-b91e-b713b014618e nodeName:}" failed. No retries permitted until 2026-05-21 17:51:00.702471535 +0000 UTC m=+500.812864511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b13d3629-00bd-42cf-b91e-b713b014618e-tls-certs") pod "kube-auth-proxy-54f55555d9-tkwmc" (UID: "b13d3629-00bd-42cf-b91e-b713b014618e") : secret "kube-auth-proxy-tls" not found
May 21 17:51:00.204854 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.204826    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b13d3629-00bd-42cf-b91e-b713b014618e-tmp\") pod \"kube-auth-proxy-54f55555d9-tkwmc\" (UID: \"b13d3629-00bd-42cf-b91e-b713b014618e\") " pod="openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc"
May 21 17:51:00.210132 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.210112    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrgzw\" (UniqueName: \"kubernetes.io/projected/b13d3629-00bd-42cf-b91e-b713b014618e-kube-api-access-qrgzw\") pod \"kube-auth-proxy-54f55555d9-tkwmc\" (UID: \"b13d3629-00bd-42cf-b91e-b713b014618e\") " pod="openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc"
May 21 17:51:00.705359 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.705309    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b13d3629-00bd-42cf-b91e-b713b014618e-tls-certs\") pod \"kube-auth-proxy-54f55555d9-tkwmc\" (UID: \"b13d3629-00bd-42cf-b91e-b713b014618e\") " pod="openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc"
May 21 17:51:00.707733 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.707710    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b13d3629-00bd-42cf-b91e-b713b014618e-tls-certs\") pod \"kube-auth-proxy-54f55555d9-tkwmc\" (UID: \"b13d3629-00bd-42cf-b91e-b713b014618e\") " pod="openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc"
May 21 17:51:00.929611 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:00.929575    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc"
May 21 17:51:01.054942 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:01.054917    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc"]
May 21 17:51:01.057346 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:51:01.057320    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb13d3629_00bd_42cf_b91e_b713b014618e.slice/crio-720077c8a481d43701b2cd2b40306a5853e45d1e43fbbb3f49326a3640f1efd4 WatchSource:0}: Error finding container 720077c8a481d43701b2cd2b40306a5853e45d1e43fbbb3f49326a3640f1efd4: Status 404 returned error can't find the container with id 720077c8a481d43701b2cd2b40306a5853e45d1e43fbbb3f49326a3640f1efd4
May 21 17:51:01.898236 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:01.898199    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc" event={"ID":"b13d3629-00bd-42cf-b91e-b713b014618e","Type":"ContainerStarted","Data":"720077c8a481d43701b2cd2b40306a5853e45d1e43fbbb3f49326a3640f1efd4"}
May 21 17:51:03.457074 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:03.457041    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-nh67r"
May 21 17:51:03.457520 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:03.457507    2558 scope.go:117] "RemoveContainer" containerID="3421a51dbede6e94397c0da12d6b888ba87b393e3fee969a54872db846c792b9"
May 21 17:51:03.457746 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:51:03.457723    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-nh67r_opendatahub(e2c025d5-a928-437c-8a6a-26f8be22402b)\"" pod="opendatahub/odh-model-controller-858dbf95b8-nh67r" podUID="e2c025d5-a928-437c-8a6a-26f8be22402b"
May 21 17:51:04.907899 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:04.907863    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-nh5d7" event={"ID":"f213e0dc-1d4d-46d6-a740-e1f44d9c2471","Type":"ContainerStarted","Data":"01d54f36e29d466394e216c5dbe5ace4a8b25677f3d2d2afb2ed12e0a208d952"}
May 21 17:51:04.908326 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:04.907955    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-856948b99f-nh5d7"
May 21 17:51:04.909112 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:04.909089    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc" event={"ID":"b13d3629-00bd-42cf-b91e-b713b014618e","Type":"ContainerStarted","Data":"ec870f76abf0ef171f97c95f58a316365fc231dd1c6017855f33783e8d68c210"}
May 21 17:51:04.923026 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:04.922985    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-856948b99f-nh5d7" podStartSLOduration=3.110565621 podStartE2EDuration="7.922974593s" podCreationTimestamp="2026-05-21 17:50:57 +0000 UTC" firstStartedPulling="2026-05-21 17:50:59.142012871 +0000 UTC m=+499.252405830" lastFinishedPulling="2026-05-21 17:51:03.954421841 +0000 UTC m=+504.064814802" observedRunningTime="2026-05-21 17:51:04.92130219 +0000 UTC m=+505.031695170" watchObservedRunningTime="2026-05-21 17:51:04.922974593 +0000 UTC m=+505.033367572"
May 21 17:51:04.935312 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:04.935270    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-54f55555d9-tkwmc" podStartSLOduration=3.036590654 podStartE2EDuration="5.935258837s" podCreationTimestamp="2026-05-21 17:50:59 +0000 UTC" firstStartedPulling="2026-05-21 17:51:01.05905786 +0000 UTC m=+501.169450817" lastFinishedPulling="2026-05-21 17:51:03.957726039 +0000 UTC m=+504.068119000" observedRunningTime="2026-05-21 17:51:04.933661982 +0000 UTC m=+505.044054974" watchObservedRunningTime="2026-05-21 17:51:04.935258837 +0000 UTC m=+505.045651816"
May 21 17:51:13.457355 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:13.457306    2558 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-858dbf95b8-nh67r"
May 21 17:51:13.457760 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:13.457714    2558 scope.go:117] "RemoveContainer" containerID="3421a51dbede6e94397c0da12d6b888ba87b393e3fee969a54872db846c792b9"
May 21 17:51:13.935087 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:13.935049    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-nh67r" event={"ID":"e2c025d5-a928-437c-8a6a-26f8be22402b","Type":"ContainerStarted","Data":"8d36028ec72d9a6297fd42cdeb051de9411cb53f32fba74943bea9e75d297b2f"}
May 21 17:51:13.935291 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:13.935269    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-nh67r"
May 21 17:51:13.962237 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:13.962188    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-858dbf95b8-nh67r" podStartSLOduration=1.771213429 podStartE2EDuration="21.962175271s" podCreationTimestamp="2026-05-21 17:50:52 +0000 UTC" firstStartedPulling="2026-05-21 17:50:53.578937411 +0000 UTC m=+493.689330370" lastFinishedPulling="2026-05-21 17:51:13.769899255 +0000 UTC m=+513.880292212" observedRunningTime="2026-05-21 17:51:13.96137446 +0000 UTC m=+514.071767440" watchObservedRunningTime="2026-05-21 17:51:13.962175271 +0000 UTC m=+514.072568251"
May 21 17:51:14.948560 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:14.948522    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-gv7x4"]
May 21 17:51:14.951822 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:14.951803    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-gv7x4"
May 21 17:51:14.954083 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:14.954058    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\""
May 21 17:51:14.954197 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:14.954099    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-6pjg2\""
May 21 17:51:14.954197 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:14.954101    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\""
May 21 17:51:14.961473 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:14.961449    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-gv7x4"]
May 21 17:51:15.016452 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:15.016419    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/75a793a5-48b2-46f1-aa95-507d97fd655b-operator-config\") pod \"servicemesh-operator3-55f49c5f94-gv7x4\" (UID: \"75a793a5-48b2-46f1-aa95-507d97fd655b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-gv7x4"
May 21 17:51:15.016642 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:15.016495    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d2l2\" (UniqueName: \"kubernetes.io/projected/75a793a5-48b2-46f1-aa95-507d97fd655b-kube-api-access-8d2l2\") pod \"servicemesh-operator3-55f49c5f94-gv7x4\" (UID: \"75a793a5-48b2-46f1-aa95-507d97fd655b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-gv7x4"
May 21 17:51:15.117604 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:15.117565    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/75a793a5-48b2-46f1-aa95-507d97fd655b-operator-config\") pod \"servicemesh-operator3-55f49c5f94-gv7x4\" (UID: \"75a793a5-48b2-46f1-aa95-507d97fd655b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-gv7x4"
May 21 17:51:15.117774 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:15.117614    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8d2l2\" (UniqueName: \"kubernetes.io/projected/75a793a5-48b2-46f1-aa95-507d97fd655b-kube-api-access-8d2l2\") pod \"servicemesh-operator3-55f49c5f94-gv7x4\" (UID: \"75a793a5-48b2-46f1-aa95-507d97fd655b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-gv7x4"
May 21 17:51:15.120131 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:15.120106    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/75a793a5-48b2-46f1-aa95-507d97fd655b-operator-config\") pod \"servicemesh-operator3-55f49c5f94-gv7x4\" (UID: \"75a793a5-48b2-46f1-aa95-507d97fd655b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-gv7x4"
May 21 17:51:15.126237 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:15.126217    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d2l2\" (UniqueName: \"kubernetes.io/projected/75a793a5-48b2-46f1-aa95-507d97fd655b-kube-api-access-8d2l2\") pod \"servicemesh-operator3-55f49c5f94-gv7x4\" (UID: \"75a793a5-48b2-46f1-aa95-507d97fd655b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-gv7x4"
May 21 17:51:15.260934 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:15.260849    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-gv7x4"
May 21 17:51:15.392482 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:15.392453    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-gv7x4"]
May 21 17:51:15.396308 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:51:15.396283    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75a793a5_48b2_46f1_aa95_507d97fd655b.slice/crio-a83f8dfe5d960cac9dae63cfdf653d461abcd2196838009621b9e774baae6d24 WatchSource:0}: Error finding container a83f8dfe5d960cac9dae63cfdf653d461abcd2196838009621b9e774baae6d24: Status 404 returned error can't find the container with id a83f8dfe5d960cac9dae63cfdf653d461abcd2196838009621b9e774baae6d24
May 21 17:51:15.942651 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:15.942614    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-gv7x4" event={"ID":"75a793a5-48b2-46f1-aa95-507d97fd655b","Type":"ContainerStarted","Data":"a83f8dfe5d960cac9dae63cfdf653d461abcd2196838009621b9e774baae6d24"}
May 21 17:51:18.954019 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:18.953974    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-gv7x4" event={"ID":"75a793a5-48b2-46f1-aa95-507d97fd655b","Type":"ContainerStarted","Data":"fd71fd6e7dc86d21edc6a66e073c5c198a5ecf300867cfa9644bd1070f97853a"}
May 21 17:51:18.954437 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:18.954035    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-gv7x4"
May 21 17:51:18.973665 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:18.973613    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-gv7x4" podStartSLOduration=2.30098254 podStartE2EDuration="4.973598059s" podCreationTimestamp="2026-05-21 17:51:14 +0000 UTC" firstStartedPulling="2026-05-21 17:51:15.398934689 +0000 UTC m=+515.509327660" lastFinishedPulling="2026-05-21 17:51:18.071550221 +0000 UTC m=+518.181943179" observedRunningTime="2026-05-21 17:51:18.97192091 +0000 UTC m=+519.082313890" watchObservedRunningTime="2026-05-21 17:51:18.973598059 +0000 UTC m=+519.083991039"
May 21 17:51:24.941205 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:24.941169    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-858dbf95b8-nh67r"
May 21 17:51:29.960063 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:29.959983    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-gv7x4"
May 21 17:51:31.554184 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.554150    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"]
May 21 17:51:31.557614 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.557598    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.561568 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.561541    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
May 21 17:51:31.561834 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.561820    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
May 21 17:51:31.562192 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.562179    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
May 21 17:51:31.562314 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.562293    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8xjd5\""
May 21 17:51:31.562546 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.562534    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
May 21 17:51:31.583622 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.583592    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"]
May 21 17:51:31.649511 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.649474    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/61fbef2c-8573-4529-9c2f-e7e166e5104c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.649511 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.649511    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/61fbef2c-8573-4529-9c2f-e7e166e5104c-local-certs\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.649756 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.649543    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/61fbef2c-8573-4529-9c2f-e7e166e5104c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.649756 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.649571    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/61fbef2c-8573-4529-9c2f-e7e166e5104c-cacerts\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.649756 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.649649    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sfpt\" (UniqueName: \"kubernetes.io/projected/61fbef2c-8573-4529-9c2f-e7e166e5104c-kube-api-access-8sfpt\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.649756 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.649691    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61fbef2c-8573-4529-9c2f-e7e166e5104c-istio-token\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.649756 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.649717    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/61fbef2c-8573-4529-9c2f-e7e166e5104c-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.751036 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.751001    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8sfpt\" (UniqueName: \"kubernetes.io/projected/61fbef2c-8573-4529-9c2f-e7e166e5104c-kube-api-access-8sfpt\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.751036 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.751038    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61fbef2c-8573-4529-9c2f-e7e166e5104c-istio-token\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.751286 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.751055    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/61fbef2c-8573-4529-9c2f-e7e166e5104c-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.751286 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.751073    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/61fbef2c-8573-4529-9c2f-e7e166e5104c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.751286 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.751101    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/61fbef2c-8573-4529-9c2f-e7e166e5104c-local-certs\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.751286 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.751154    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/61fbef2c-8573-4529-9c2f-e7e166e5104c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.751286 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.751188    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/61fbef2c-8573-4529-9c2f-e7e166e5104c-cacerts\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.751752 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.751721    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/61fbef2c-8573-4529-9c2f-e7e166e5104c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.753559 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.753537    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/61fbef2c-8573-4529-9c2f-e7e166e5104c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.753648 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.753597    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/61fbef2c-8573-4529-9c2f-e7e166e5104c-local-certs\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.753979 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.753960    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/61fbef2c-8573-4529-9c2f-e7e166e5104c-cacerts\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.754048 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.753968    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/61fbef2c-8573-4529-9c2f-e7e166e5104c-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.758629 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.758608    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61fbef2c-8573-4529-9c2f-e7e166e5104c-istio-token\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.759333 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.759312    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sfpt\" (UniqueName: \"kubernetes.io/projected/61fbef2c-8573-4529-9c2f-e7e166e5104c-kube-api-access-8sfpt\") pod \"istiod-openshift-gateway-798958bb55-4tjtk\" (UID: \"61fbef2c-8573-4529-9c2f-e7e166e5104c\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.867191 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.867095    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:31.992756 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:31.992724    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"]
May 21 17:51:31.996534 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:51:31.996502    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61fbef2c_8573_4529_9c2f_e7e166e5104c.slice/crio-84d7c503384b46204bf489b1d2fd80a68508da22096b24f6bfea729e3441362b WatchSource:0}: Error finding container 84d7c503384b46204bf489b1d2fd80a68508da22096b24f6bfea729e3441362b: Status 404 returned error can't find the container with id 84d7c503384b46204bf489b1d2fd80a68508da22096b24f6bfea729e3441362b
May 21 17:51:32.998071 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:32.998035    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk" event={"ID":"61fbef2c-8573-4529-9c2f-e7e166e5104c","Type":"ContainerStarted","Data":"84d7c503384b46204bf489b1d2fd80a68508da22096b24f6bfea729e3441362b"}
May 21 17:51:35.226765 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:35.226728    2558 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
May 21 17:51:35.227078 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:35.226817    2558 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
May 21 17:51:35.917559 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:35.917524    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-856948b99f-nh5d7"
May 21 17:51:36.010523 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:36.010482    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk" event={"ID":"61fbef2c-8573-4529-9c2f-e7e166e5104c","Type":"ContainerStarted","Data":"598bf40e82d300b5645bb6ead4fde95d9d49278cff22f2a193b32c9b3f6ab72e"}
May 21 17:51:36.010719 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:36.010700    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:51:36.012034 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:36.012010    2558 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-4tjtk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
May 21 17:51:36.012114 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:36.012066    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk" podUID="61fbef2c-8573-4529-9c2f-e7e166e5104c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 21 17:51:36.030031 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:36.029977    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk" podStartSLOduration=1.8017531359999999 podStartE2EDuration="5.029961724s" podCreationTimestamp="2026-05-21 17:51:31 +0000 UTC" firstStartedPulling="2026-05-21 17:51:31.998304875 +0000 UTC m=+532.108697842" lastFinishedPulling="2026-05-21 17:51:35.226513472 +0000 UTC m=+535.336906430" observedRunningTime="2026-05-21 17:51:36.028959339 +0000 UTC m=+536.139352316" watchObservedRunningTime="2026-05-21 17:51:36.029961724 +0000 UTC m=+536.140354706"
May 21 17:51:37.014866 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:51:37.014838    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4tjtk"
May 21 17:52:26.733280 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:26.733247    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"]
May 21 17:52:26.742986 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:26.742959    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"
May 21 17:52:26.745760 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:26.745737    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6psh9\""
May 21 17:52:26.745897 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:26.745761    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
May 21 17:52:26.745897 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:26.745762    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
May 21 17:52:26.748519 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:26.748498    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"]
May 21 17:52:26.785748 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:26.785716    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b4ea44c-02cb-4cb9-8696-adcd16cc6698-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j5k2q\" (UID: \"2b4ea44c-02cb-4cb9-8696-adcd16cc6698\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"
May 21 17:52:26.785910 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:26.785757    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhs6n\" (UniqueName: \"kubernetes.io/projected/2b4ea44c-02cb-4cb9-8696-adcd16cc6698-kube-api-access-mhs6n\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j5k2q\" (UID: \"2b4ea44c-02cb-4cb9-8696-adcd16cc6698\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"
May 21 17:52:26.887161 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:26.887129    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mhs6n\" (UniqueName: \"kubernetes.io/projected/2b4ea44c-02cb-4cb9-8696-adcd16cc6698-kube-api-access-mhs6n\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j5k2q\" (UID: \"2b4ea44c-02cb-4cb9-8696-adcd16cc6698\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"
May 21 17:52:26.887371 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:26.887199    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b4ea44c-02cb-4cb9-8696-adcd16cc6698-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j5k2q\" (UID: \"2b4ea44c-02cb-4cb9-8696-adcd16cc6698\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"
May 21 17:52:26.887527 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:26.887509    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b4ea44c-02cb-4cb9-8696-adcd16cc6698-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j5k2q\" (UID: \"2b4ea44c-02cb-4cb9-8696-adcd16cc6698\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"
May 21 17:52:26.902148 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:26.902115    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhs6n\" (UniqueName: \"kubernetes.io/projected/2b4ea44c-02cb-4cb9-8696-adcd16cc6698-kube-api-access-mhs6n\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j5k2q\" (UID: \"2b4ea44c-02cb-4cb9-8696-adcd16cc6698\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"
May 21 17:52:27.053921 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:27.053836    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"
May 21 17:52:27.185240 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:27.185028    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"]
May 21 17:52:27.188555 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:52:27.188530    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b4ea44c_02cb_4cb9_8696_adcd16cc6698.slice/crio-5799a5223ce5c7febf5229411767ed11c37c18e50676e0304a3dd9d020ff13a8 WatchSource:0}: Error finding container 5799a5223ce5c7febf5229411767ed11c37c18e50676e0304a3dd9d020ff13a8: Status 404 returned error can't find the container with id 5799a5223ce5c7febf5229411767ed11c37c18e50676e0304a3dd9d020ff13a8
May 21 17:52:28.167658 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:28.167619    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q" event={"ID":"2b4ea44c-02cb-4cb9-8696-adcd16cc6698","Type":"ContainerStarted","Data":"5799a5223ce5c7febf5229411767ed11c37c18e50676e0304a3dd9d020ff13a8"}
May 21 17:52:33.184926 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:33.184885    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q" event={"ID":"2b4ea44c-02cb-4cb9-8696-adcd16cc6698","Type":"ContainerStarted","Data":"1dd4560dd1ca6cd1b61684a2b4ac26bb7f03cc30ba2ed6a4ea525715513299ed"}
May 21 17:52:33.185334 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:33.185121    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"
May 21 17:52:33.203114 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:33.203061    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q" podStartSLOduration=1.84886494 podStartE2EDuration="7.203044366s" podCreationTimestamp="2026-05-21 17:52:26 +0000 UTC" firstStartedPulling="2026-05-21 17:52:27.190831771 +0000 UTC m=+587.301224730" lastFinishedPulling="2026-05-21 17:52:32.545011195 +0000 UTC m=+592.655404156" observedRunningTime="2026-05-21 17:52:33.20169798 +0000 UTC m=+593.312090960" watchObservedRunningTime="2026-05-21 17:52:33.203044366 +0000 UTC m=+593.313437347"
May 21 17:52:40.412637 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:40.412608    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 17:52:40.413809 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:40.413772    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 17:52:44.191091 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:44.191060    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"
May 21 17:52:45.196652 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.196617    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw"]
May 21 17:52:45.200202 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.200185    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw"
May 21 17:52:45.212232 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.212208    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw"]
May 21 17:52:45.336486 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.336448    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxcqx\" (UniqueName: \"kubernetes.io/projected/f09d8331-8a7a-4d48-94fd-564daa811951-kube-api-access-wxcqx\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-9jspw\" (UID: \"f09d8331-8a7a-4d48-94fd-564daa811951\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw"
May 21 17:52:45.336655 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.336506    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f09d8331-8a7a-4d48-94fd-564daa811951-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-9jspw\" (UID: \"f09d8331-8a7a-4d48-94fd-564daa811951\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw"
May 21 17:52:45.437421 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.437382    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wxcqx\" (UniqueName: \"kubernetes.io/projected/f09d8331-8a7a-4d48-94fd-564daa811951-kube-api-access-wxcqx\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-9jspw\" (UID: \"f09d8331-8a7a-4d48-94fd-564daa811951\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw"
May 21 17:52:45.437595 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.437432    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f09d8331-8a7a-4d48-94fd-564daa811951-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-9jspw\" (UID: \"f09d8331-8a7a-4d48-94fd-564daa811951\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw"
May 21 17:52:45.437760 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.437743    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f09d8331-8a7a-4d48-94fd-564daa811951-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-9jspw\" (UID: \"f09d8331-8a7a-4d48-94fd-564daa811951\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw"
May 21 17:52:45.445181 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.445155    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxcqx\" (UniqueName: \"kubernetes.io/projected/f09d8331-8a7a-4d48-94fd-564daa811951-kube-api-access-wxcqx\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-9jspw\" (UID: \"f09d8331-8a7a-4d48-94fd-564daa811951\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw"
May 21 17:52:45.510351 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.510256    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw"
May 21 17:52:45.642797 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.642757    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw"]
May 21 17:52:45.645907 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:52:45.645881    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf09d8331_8a7a_4d48_94fd_564daa811951.slice/crio-5abb642c7b3197d2db5622a41cb5e8e17f3a11f1f7efb73f952756fafe772360 WatchSource:0}: Error finding container 5abb642c7b3197d2db5622a41cb5e8e17f3a11f1f7efb73f952756fafe772360: Status 404 returned error can't find the container with id 5abb642c7b3197d2db5622a41cb5e8e17f3a11f1f7efb73f952756fafe772360
May 21 17:52:45.843067 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.842981    2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"]
May 21 17:52:45.843287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.843263    2558 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q" podUID="2b4ea44c-02cb-4cb9-8696-adcd16cc6698" containerName="manager" containerID="cri-o://1dd4560dd1ca6cd1b61684a2b4ac26bb7f03cc30ba2ed6a4ea525715513299ed" gracePeriod=2
May 21 17:52:45.853519 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.853492    2558 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"]
May 21 17:52:45.864932 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.864905    2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw"]
May 21 17:52:45.866088 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.866068    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"]
May 21 17:52:45.866354 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.866341    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b4ea44c-02cb-4cb9-8696-adcd16cc6698" containerName="manager"
May 21 17:52:45.866396 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.866356    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b4ea44c-02cb-4cb9-8696-adcd16cc6698" containerName="manager"
May 21 17:52:45.866433 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.866414    2558 memory_manager.go:356] "RemoveStaleState removing state" podUID="2b4ea44c-02cb-4cb9-8696-adcd16cc6698" containerName="manager"
May 21 17:52:45.869522 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.869487    2558 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw"]
May 21 17:52:45.869650 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.869596    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"
May 21 17:52:45.871559 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.871527    2558 status_manager.go:895] "Failed to get status for pod" podUID="2b4ea44c-02cb-4cb9-8696-adcd16cc6698" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-j5k2q\" is forbidden: User \"system:node:ip-10-0-135-125.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-125.ec2.internal' and this object"
May 21 17:52:45.881265 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.881238    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"]
May 21 17:52:45.889056 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.889021    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"]
May 21 17:52:45.889308 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.889296    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f09d8331-8a7a-4d48-94fd-564daa811951" containerName="manager"
May 21 17:52:45.889356 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.889310    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f09d8331-8a7a-4d48-94fd-564daa811951" containerName="manager"
May 21 17:52:45.889393 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.889370    2558 memory_manager.go:356] "RemoveStaleState removing state" podUID="f09d8331-8a7a-4d48-94fd-564daa811951" containerName="manager"
May 21 17:52:45.892150 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.892129    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"
May 21 17:52:45.918933 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.918622    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"]
May 21 17:52:45.931299 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.931265    2558 status_manager.go:895] "Failed to get status for pod" podUID="2b4ea44c-02cb-4cb9-8696-adcd16cc6698" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-j5k2q\" is forbidden: User \"system:node:ip-10-0-135-125.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-125.ec2.internal' and this object"
May 21 17:52:45.941729 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.941688    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xpps\" (UniqueName: \"kubernetes.io/projected/23e46708-7b6e-47ab-bfe5-569eec930cb3-kube-api-access-9xpps\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-z4nmd\" (UID: \"23e46708-7b6e-47ab-bfe5-569eec930cb3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"
May 21 17:52:45.941934 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:45.941909    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23e46708-7b6e-47ab-bfe5-569eec930cb3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-z4nmd\" (UID: \"23e46708-7b6e-47ab-bfe5-569eec930cb3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"
May 21 17:52:46.043331 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.043297    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flnqj\" (UniqueName: \"kubernetes.io/projected/7b2062b9-2ade-4b61-a486-53678a93fc79-kube-api-access-flnqj\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w\" (UID: \"7b2062b9-2ade-4b61-a486-53678a93fc79\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"
May 21 17:52:46.043528 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.043350    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23e46708-7b6e-47ab-bfe5-569eec930cb3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-z4nmd\" (UID: \"23e46708-7b6e-47ab-bfe5-569eec930cb3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"
May 21 17:52:46.043528 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.043464    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b2062b9-2ade-4b61-a486-53678a93fc79-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w\" (UID: \"7b2062b9-2ade-4b61-a486-53678a93fc79\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"
May 21 17:52:46.043614 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.043520    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9xpps\" (UniqueName: \"kubernetes.io/projected/23e46708-7b6e-47ab-bfe5-569eec930cb3-kube-api-access-9xpps\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-z4nmd\" (UID: \"23e46708-7b6e-47ab-bfe5-569eec930cb3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"
May 21 17:52:46.043734 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.043717    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23e46708-7b6e-47ab-bfe5-569eec930cb3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-z4nmd\" (UID: \"23e46708-7b6e-47ab-bfe5-569eec930cb3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"
May 21 17:52:46.050899 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.050877    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xpps\" (UniqueName: \"kubernetes.io/projected/23e46708-7b6e-47ab-bfe5-569eec930cb3-kube-api-access-9xpps\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-z4nmd\" (UID: \"23e46708-7b6e-47ab-bfe5-569eec930cb3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"
May 21 17:52:46.068370 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.068350    2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"
May 21 17:52:46.070202 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.070175    2558 status_manager.go:895] "Failed to get status for pod" podUID="2b4ea44c-02cb-4cb9-8696-adcd16cc6698" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-j5k2q\" is forbidden: User \"system:node:ip-10-0-135-125.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-125.ec2.internal' and this object"
May 21 17:52:46.144739 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.144645    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b4ea44c-02cb-4cb9-8696-adcd16cc6698-extensions-socket-volume\") pod \"2b4ea44c-02cb-4cb9-8696-adcd16cc6698\" (UID: \"2b4ea44c-02cb-4cb9-8696-adcd16cc6698\") "
May 21 17:52:46.144739 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.144739    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhs6n\" (UniqueName: \"kubernetes.io/projected/2b4ea44c-02cb-4cb9-8696-adcd16cc6698-kube-api-access-mhs6n\") pod \"2b4ea44c-02cb-4cb9-8696-adcd16cc6698\" (UID: \"2b4ea44c-02cb-4cb9-8696-adcd16cc6698\") "
May 21 17:52:46.144978 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.144919    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b2062b9-2ade-4b61-a486-53678a93fc79-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w\" (UID: \"7b2062b9-2ade-4b61-a486-53678a93fc79\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"
May 21 17:52:46.145033 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.145019    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-flnqj\" (UniqueName: \"kubernetes.io/projected/7b2062b9-2ade-4b61-a486-53678a93fc79-kube-api-access-flnqj\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w\" (UID: \"7b2062b9-2ade-4b61-a486-53678a93fc79\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"
May 21 17:52:46.145105 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.145071    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b4ea44c-02cb-4cb9-8696-adcd16cc6698-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2b4ea44c-02cb-4cb9-8696-adcd16cc6698" (UID: "2b4ea44c-02cb-4cb9-8696-adcd16cc6698"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 21 17:52:46.145296 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.145278    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b2062b9-2ade-4b61-a486-53678a93fc79-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w\" (UID: \"7b2062b9-2ade-4b61-a486-53678a93fc79\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"
May 21 17:52:46.146893 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.146871    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b4ea44c-02cb-4cb9-8696-adcd16cc6698-kube-api-access-mhs6n" (OuterVolumeSpecName: "kube-api-access-mhs6n") pod "2b4ea44c-02cb-4cb9-8696-adcd16cc6698" (UID: "2b4ea44c-02cb-4cb9-8696-adcd16cc6698"). InnerVolumeSpecName "kube-api-access-mhs6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 17:52:46.154185 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.154160    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flnqj\" (UniqueName: \"kubernetes.io/projected/7b2062b9-2ade-4b61-a486-53678a93fc79-kube-api-access-flnqj\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w\" (UID: \"7b2062b9-2ade-4b61-a486-53678a93fc79\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"
May 21 17:52:46.219923 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.219893    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"
May 21 17:52:46.226425 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.226397    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"
May 21 17:52:46.228087 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.228062    2558 generic.go:358] "Generic (PLEG): container finished" podID="2b4ea44c-02cb-4cb9-8696-adcd16cc6698" containerID="1dd4560dd1ca6cd1b61684a2b4ac26bb7f03cc30ba2ed6a4ea525715513299ed" exitCode=0
May 21 17:52:46.228199 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.228105    2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q"
May 21 17:52:46.228199 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.228175    2558 scope.go:117] "RemoveContainer" containerID="1dd4560dd1ca6cd1b61684a2b4ac26bb7f03cc30ba2ed6a4ea525715513299ed"
May 21 17:52:46.230418 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.230387    2558 status_manager.go:895] "Failed to get status for pod" podUID="2b4ea44c-02cb-4cb9-8696-adcd16cc6698" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-j5k2q\" is forbidden: User \"system:node:ip-10-0-135-125.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-125.ec2.internal' and this object"
May 21 17:52:46.230548 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:52:46.230438    2558 kuberuntime_manager.go:623] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw" container="manager"
May 21 17:52:46.232033 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.232002    2558 status_manager.go:895] "Failed to get status for pod" podUID="2b4ea44c-02cb-4cb9-8696-adcd16cc6698" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-j5k2q\" is forbidden: User \"system:node:ip-10-0-135-125.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-125.ec2.internal' and this object"
May 21 17:52:46.233899 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.233862    2558 status_manager.go:895] "Failed to get status for pod" podUID="f09d8331-8a7a-4d48-94fd-564daa811951" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-9jspw\" is forbidden: User \"system:node:ip-10-0-135-125.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-125.ec2.internal' and this object"
May 21 17:52:46.238103 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.238078    2558 scope.go:117] "RemoveContainer" containerID="1dd4560dd1ca6cd1b61684a2b4ac26bb7f03cc30ba2ed6a4ea525715513299ed"
May 21 17:52:46.238394 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:52:46.238370    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dd4560dd1ca6cd1b61684a2b4ac26bb7f03cc30ba2ed6a4ea525715513299ed\": container with ID starting with 1dd4560dd1ca6cd1b61684a2b4ac26bb7f03cc30ba2ed6a4ea525715513299ed not found: ID does not exist" containerID="1dd4560dd1ca6cd1b61684a2b4ac26bb7f03cc30ba2ed6a4ea525715513299ed"
May 21 17:52:46.238478 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.238404    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dd4560dd1ca6cd1b61684a2b4ac26bb7f03cc30ba2ed6a4ea525715513299ed"} err="failed to get container status \"1dd4560dd1ca6cd1b61684a2b4ac26bb7f03cc30ba2ed6a4ea525715513299ed\": rpc error: code = NotFound desc = could not find container \"1dd4560dd1ca6cd1b61684a2b4ac26bb7f03cc30ba2ed6a4ea525715513299ed\": container with ID starting with 1dd4560dd1ca6cd1b61684a2b4ac26bb7f03cc30ba2ed6a4ea525715513299ed not found: ID does not exist"
May 21 17:52:46.241287 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.241260    2558 status_manager.go:895] "Failed to get status for pod" podUID="f09d8331-8a7a-4d48-94fd-564daa811951" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-9jspw\" is forbidden: User \"system:node:ip-10-0-135-125.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-125.ec2.internal' and this object"
May 21 17:52:46.242964 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.242936    2558 status_manager.go:895] "Failed to get status for pod" podUID="2b4ea44c-02cb-4cb9-8696-adcd16cc6698" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j5k2q" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-j5k2q\" is forbidden: User \"system:node:ip-10-0-135-125.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-125.ec2.internal' and this object"
May 21 17:52:46.245661 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.245638    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhs6n\" (UniqueName: \"kubernetes.io/projected/2b4ea44c-02cb-4cb9-8696-adcd16cc6698-kube-api-access-mhs6n\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:52:46.245764 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.245666    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b4ea44c-02cb-4cb9-8696-adcd16cc6698-extensions-socket-volume\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:52:46.361770 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.361736    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"]
May 21 17:52:46.366030 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:52:46.365995    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23e46708_7b6e_47ab_bfe5_569eec930cb3.slice/crio-fe5c65ee3fbe31f6745daf1989fac250a8ed2f03289a8f29e517f95d079c8456 WatchSource:0}: Error finding container fe5c65ee3fbe31f6745daf1989fac250a8ed2f03289a8f29e517f95d079c8456: Status 404 returned error can't find the container with id fe5c65ee3fbe31f6745daf1989fac250a8ed2f03289a8f29e517f95d079c8456
May 21 17:52:46.383241 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.383219    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"]
May 21 17:52:46.386268 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:52:46.386239    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b2062b9_2ade_4b61_a486_53678a93fc79.slice/crio-7fc8d1d797298eeec9a8d74cf90e4a414b5e0874010f83f2d77fd54e66519491 WatchSource:0}: Error finding container 7fc8d1d797298eeec9a8d74cf90e4a414b5e0874010f83f2d77fd54e66519491: Status 404 returned error can't find the container with id 7fc8d1d797298eeec9a8d74cf90e4a414b5e0874010f83f2d77fd54e66519491
May 21 17:52:46.507613 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:46.507578    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b4ea44c-02cb-4cb9-8696-adcd16cc6698" path="/var/lib/kubelet/pods/2b4ea44c-02cb-4cb9-8696-adcd16cc6698/volumes"
May 21 17:52:47.235474 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.235435    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd" event={"ID":"23e46708-7b6e-47ab-bfe5-569eec930cb3","Type":"ContainerStarted","Data":"3d8074fbb9910d232d4fd8d09039a2b6840f9aea08946a15fb304f94cf366b89"}
May 21 17:52:47.235474 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.235476    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd" event={"ID":"23e46708-7b6e-47ab-bfe5-569eec930cb3","Type":"ContainerStarted","Data":"fe5c65ee3fbe31f6745daf1989fac250a8ed2f03289a8f29e517f95d079c8456"}
May 21 17:52:47.235985 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.235506    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"
May 21 17:52:47.236774 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.236754    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w" event={"ID":"7b2062b9-2ade-4b61-a486-53678a93fc79","Type":"ContainerStarted","Data":"d7c2968383d624941d6b1df15ac6fe6bc8edd63902175d0908ab01ad81f8d2a5"}
May 21 17:52:47.236899 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.236778    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w" event={"ID":"7b2062b9-2ade-4b61-a486-53678a93fc79","Type":"ContainerStarted","Data":"7fc8d1d797298eeec9a8d74cf90e4a414b5e0874010f83f2d77fd54e66519491"}
May 21 17:52:47.236899 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.236866    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"
May 21 17:52:47.238155 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.238131    2558 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw" podUID="f09d8331-8a7a-4d48-94fd-564daa811951" containerName="manager" containerID="cri-o://1ddde556e49e91c1013d9ad87e55d9e47b16f17576e52afd11539e145df13262" gracePeriod=2
May 21 17:52:47.278701 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.278653    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd" podStartSLOduration=2.278640233 podStartE2EDuration="2.278640233s" podCreationTimestamp="2026-05-21 17:52:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 17:52:47.277398252 +0000 UTC m=+607.387791232" watchObservedRunningTime="2026-05-21 17:52:47.278640233 +0000 UTC m=+607.389033213"
May 21 17:52:47.309001 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.308958    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w" podStartSLOduration=2.308944179 podStartE2EDuration="2.308944179s" podCreationTimestamp="2026-05-21 17:52:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 17:52:47.306956744 +0000 UTC m=+607.417349724" watchObservedRunningTime="2026-05-21 17:52:47.308944179 +0000 UTC m=+607.419337197"
May 21 17:52:47.464339 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.464316    2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw"
May 21 17:52:47.466158 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.466133    2558 status_manager.go:895] "Failed to get status for pod" podUID="f09d8331-8a7a-4d48-94fd-564daa811951" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-9jspw\" is forbidden: User \"system:node:ip-10-0-135-125.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-125.ec2.internal' and this object"
May 21 17:52:47.558626 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.558542    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f09d8331-8a7a-4d48-94fd-564daa811951-extensions-socket-volume\") pod \"f09d8331-8a7a-4d48-94fd-564daa811951\" (UID: \"f09d8331-8a7a-4d48-94fd-564daa811951\") "
May 21 17:52:47.558626 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.558589    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxcqx\" (UniqueName: \"kubernetes.io/projected/f09d8331-8a7a-4d48-94fd-564daa811951-kube-api-access-wxcqx\") pod \"f09d8331-8a7a-4d48-94fd-564daa811951\" (UID: \"f09d8331-8a7a-4d48-94fd-564daa811951\") "
May 21 17:52:47.558829 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.558806    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f09d8331-8a7a-4d48-94fd-564daa811951-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f09d8331-8a7a-4d48-94fd-564daa811951" (UID: "f09d8331-8a7a-4d48-94fd-564daa811951"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 21 17:52:47.560608 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.560588    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f09d8331-8a7a-4d48-94fd-564daa811951-kube-api-access-wxcqx" (OuterVolumeSpecName: "kube-api-access-wxcqx") pod "f09d8331-8a7a-4d48-94fd-564daa811951" (UID: "f09d8331-8a7a-4d48-94fd-564daa811951"). InnerVolumeSpecName "kube-api-access-wxcqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 17:52:47.659854 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.659818    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f09d8331-8a7a-4d48-94fd-564daa811951-extensions-socket-volume\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:52:47.659854 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:47.659846    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxcqx\" (UniqueName: \"kubernetes.io/projected/f09d8331-8a7a-4d48-94fd-564daa811951-kube-api-access-wxcqx\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:52:48.243586 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:48.243550    2558 generic.go:358] "Generic (PLEG): container finished" podID="f09d8331-8a7a-4d48-94fd-564daa811951" containerID="1ddde556e49e91c1013d9ad87e55d9e47b16f17576e52afd11539e145df13262" exitCode=2
May 21 17:52:48.244142 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:48.243619    2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw"
May 21 17:52:48.244142 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:48.243652    2558 scope.go:117] "RemoveContainer" containerID="1ddde556e49e91c1013d9ad87e55d9e47b16f17576e52afd11539e145df13262"
May 21 17:52:48.247715 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:48.247684    2558 status_manager.go:895] "Failed to get status for pod" podUID="f09d8331-8a7a-4d48-94fd-564daa811951" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-9jspw\" is forbidden: User \"system:node:ip-10-0-135-125.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-125.ec2.internal' and this object"
May 21 17:52:48.251402 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:48.251362    2558 scope.go:117] "RemoveContainer" containerID="1ddde556e49e91c1013d9ad87e55d9e47b16f17576e52afd11539e145df13262"
May 21 17:52:48.251638 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:52:48.251615    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ddde556e49e91c1013d9ad87e55d9e47b16f17576e52afd11539e145df13262\": container with ID starting with 1ddde556e49e91c1013d9ad87e55d9e47b16f17576e52afd11539e145df13262 not found: ID does not exist" containerID="1ddde556e49e91c1013d9ad87e55d9e47b16f17576e52afd11539e145df13262"
May 21 17:52:48.251689 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:48.251646    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ddde556e49e91c1013d9ad87e55d9e47b16f17576e52afd11539e145df13262"} err="failed to get container status \"1ddde556e49e91c1013d9ad87e55d9e47b16f17576e52afd11539e145df13262\": rpc error: code = NotFound desc = could not find container \"1ddde556e49e91c1013d9ad87e55d9e47b16f17576e52afd11539e145df13262\": container with ID starting with 1ddde556e49e91c1013d9ad87e55d9e47b16f17576e52afd11539e145df13262 not found: ID does not exist"
May 21 17:52:48.253821 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:48.253801    2558 status_manager.go:895] "Failed to get status for pod" podUID="f09d8331-8a7a-4d48-94fd-564daa811951" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jspw" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-9jspw\" is forbidden: User \"system:node:ip-10-0-135-125.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-125.ec2.internal' and this object"
May 21 17:52:48.506959 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:48.506881    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f09d8331-8a7a-4d48-94fd-564daa811951" path="/var/lib/kubelet/pods/f09d8331-8a7a-4d48-94fd-564daa811951/volumes"
May 21 17:52:58.246431 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.246348    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"
May 21 17:52:58.246808 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.246525    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"
May 21 17:52:58.307472 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.307432    2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"]
May 21 17:52:58.307695 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.307651    2558 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd" podUID="23e46708-7b6e-47ab-bfe5-569eec930cb3" containerName="manager" containerID="cri-o://3d8074fbb9910d232d4fd8d09039a2b6840f9aea08946a15fb304f94cf366b89" gracePeriod=10
May 21 17:52:58.509472 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.509398    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"]
May 21 17:52:58.512775 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.512757    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"
May 21 17:52:58.522536 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.522511    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"]
May 21 17:52:58.563140 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.563117    2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"
May 21 17:52:58.646637 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.646601    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23e46708-7b6e-47ab-bfe5-569eec930cb3-extensions-socket-volume\") pod \"23e46708-7b6e-47ab-bfe5-569eec930cb3\" (UID: \"23e46708-7b6e-47ab-bfe5-569eec930cb3\") "
May 21 17:52:58.646840 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.646670    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xpps\" (UniqueName: \"kubernetes.io/projected/23e46708-7b6e-47ab-bfe5-569eec930cb3-kube-api-access-9xpps\") pod \"23e46708-7b6e-47ab-bfe5-569eec930cb3\" (UID: \"23e46708-7b6e-47ab-bfe5-569eec930cb3\") "
May 21 17:52:58.646908 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.646852    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c29ae5f9-c77b-4c59-8afb-2b38d6be71cb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-9glsd\" (UID: \"c29ae5f9-c77b-4c59-8afb-2b38d6be71cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"
May 21 17:52:58.646968 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.646950    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4bv7\" (UniqueName: \"kubernetes.io/projected/c29ae5f9-c77b-4c59-8afb-2b38d6be71cb-kube-api-access-f4bv7\") pod \"kuadrant-operator-controller-manager-55c7f4c975-9glsd\" (UID: \"c29ae5f9-c77b-4c59-8afb-2b38d6be71cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"
May 21 17:52:58.646968 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.646956    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23e46708-7b6e-47ab-bfe5-569eec930cb3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "23e46708-7b6e-47ab-bfe5-569eec930cb3" (UID: "23e46708-7b6e-47ab-bfe5-569eec930cb3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 21 17:52:58.647164 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.647144    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23e46708-7b6e-47ab-bfe5-569eec930cb3-extensions-socket-volume\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:52:58.648625 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.648606    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23e46708-7b6e-47ab-bfe5-569eec930cb3-kube-api-access-9xpps" (OuterVolumeSpecName: "kube-api-access-9xpps") pod "23e46708-7b6e-47ab-bfe5-569eec930cb3" (UID: "23e46708-7b6e-47ab-bfe5-569eec930cb3"). InnerVolumeSpecName "kube-api-access-9xpps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 17:52:58.747838 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.747780    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c29ae5f9-c77b-4c59-8afb-2b38d6be71cb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-9glsd\" (UID: \"c29ae5f9-c77b-4c59-8afb-2b38d6be71cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"
May 21 17:52:58.747993 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.747867    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f4bv7\" (UniqueName: \"kubernetes.io/projected/c29ae5f9-c77b-4c59-8afb-2b38d6be71cb-kube-api-access-f4bv7\") pod \"kuadrant-operator-controller-manager-55c7f4c975-9glsd\" (UID: \"c29ae5f9-c77b-4c59-8afb-2b38d6be71cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"
May 21 17:52:58.747993 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.747947    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xpps\" (UniqueName: \"kubernetes.io/projected/23e46708-7b6e-47ab-bfe5-569eec930cb3-kube-api-access-9xpps\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:52:58.748127 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.748107    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c29ae5f9-c77b-4c59-8afb-2b38d6be71cb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-9glsd\" (UID: \"c29ae5f9-c77b-4c59-8afb-2b38d6be71cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"
May 21 17:52:58.755479 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.755460    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4bv7\" (UniqueName: \"kubernetes.io/projected/c29ae5f9-c77b-4c59-8afb-2b38d6be71cb-kube-api-access-f4bv7\") pod \"kuadrant-operator-controller-manager-55c7f4c975-9glsd\" (UID: \"c29ae5f9-c77b-4c59-8afb-2b38d6be71cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"
May 21 17:52:58.823988 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.823908    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"
May 21 17:52:58.952158 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:58.952123    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"]
May 21 17:52:58.954248 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:52:58.954221    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc29ae5f9_c77b_4c59_8afb_2b38d6be71cb.slice/crio-ae118587732edb98ea6c009324cabee53fa7a85afb4227f8ed44355335547c5d WatchSource:0}: Error finding container ae118587732edb98ea6c009324cabee53fa7a85afb4227f8ed44355335547c5d: Status 404 returned error can't find the container with id ae118587732edb98ea6c009324cabee53fa7a85afb4227f8ed44355335547c5d
May 21 17:52:59.283160 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:59.283127    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd" event={"ID":"c29ae5f9-c77b-4c59-8afb-2b38d6be71cb","Type":"ContainerStarted","Data":"ad102817c5dab5046f9e67cb8e7dd4e1557ffaf11245b30edcc528332f422320"}
May 21 17:52:59.283160 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:59.283165    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd" event={"ID":"c29ae5f9-c77b-4c59-8afb-2b38d6be71cb","Type":"ContainerStarted","Data":"ae118587732edb98ea6c009324cabee53fa7a85afb4227f8ed44355335547c5d"}
May 21 17:52:59.283609 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:59.283197    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"
May 21 17:52:59.284264 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:59.284240    2558 generic.go:358] "Generic (PLEG): container finished" podID="23e46708-7b6e-47ab-bfe5-569eec930cb3" containerID="3d8074fbb9910d232d4fd8d09039a2b6840f9aea08946a15fb304f94cf366b89" exitCode=0
May 21 17:52:59.284368 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:59.284294    2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"
May 21 17:52:59.284368 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:59.284299    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd" event={"ID":"23e46708-7b6e-47ab-bfe5-569eec930cb3","Type":"ContainerDied","Data":"3d8074fbb9910d232d4fd8d09039a2b6840f9aea08946a15fb304f94cf366b89"}
May 21 17:52:59.284368 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:59.284321    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd" event={"ID":"23e46708-7b6e-47ab-bfe5-569eec930cb3","Type":"ContainerDied","Data":"fe5c65ee3fbe31f6745daf1989fac250a8ed2f03289a8f29e517f95d079c8456"}
May 21 17:52:59.284368 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:59.284336    2558 scope.go:117] "RemoveContainer" containerID="3d8074fbb9910d232d4fd8d09039a2b6840f9aea08946a15fb304f94cf366b89"
May 21 17:52:59.292780 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:59.292764    2558 scope.go:117] "RemoveContainer" containerID="3d8074fbb9910d232d4fd8d09039a2b6840f9aea08946a15fb304f94cf366b89"
May 21 17:52:59.293049 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:52:59.293030    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d8074fbb9910d232d4fd8d09039a2b6840f9aea08946a15fb304f94cf366b89\": container with ID starting with 3d8074fbb9910d232d4fd8d09039a2b6840f9aea08946a15fb304f94cf366b89 not found: ID does not exist" containerID="3d8074fbb9910d232d4fd8d09039a2b6840f9aea08946a15fb304f94cf366b89"
May 21 17:52:59.293094 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:59.293056    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d8074fbb9910d232d4fd8d09039a2b6840f9aea08946a15fb304f94cf366b89"} err="failed to get container status \"3d8074fbb9910d232d4fd8d09039a2b6840f9aea08946a15fb304f94cf366b89\": rpc error: code = NotFound desc = could not find container \"3d8074fbb9910d232d4fd8d09039a2b6840f9aea08946a15fb304f94cf366b89\": container with ID starting with 3d8074fbb9910d232d4fd8d09039a2b6840f9aea08946a15fb304f94cf366b89 not found: ID does not exist"
May 21 17:52:59.300568 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:59.300531    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd" podStartSLOduration=1.3005202009999999 podStartE2EDuration="1.300520201s" podCreationTimestamp="2026-05-21 17:52:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 17:52:59.298715576 +0000 UTC m=+619.409108557" watchObservedRunningTime="2026-05-21 17:52:59.300520201 +0000 UTC m=+619.410913181"
May 21 17:52:59.314616 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:59.314587    2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"]
May 21 17:52:59.325184 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:52:59.325147    2558 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-z4nmd"]
May 21 17:53:00.506916 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:00.506884    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23e46708-7b6e-47ab-bfe5-569eec930cb3" path="/var/lib/kubelet/pods/23e46708-7b6e-47ab-bfe5-569eec930cb3/volumes"
May 21 17:53:10.291252 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:10.291220    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"
May 21 17:53:10.339868 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:10.339833    2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"]
May 21 17:53:10.340070 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:10.340047    2558 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w" podUID="7b2062b9-2ade-4b61-a486-53678a93fc79" containerName="manager" containerID="cri-o://d7c2968383d624941d6b1df15ac6fe6bc8edd63902175d0908ab01ad81f8d2a5" gracePeriod=10
May 21 17:53:10.588668 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:10.588646    2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"
May 21 17:53:10.642751 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:10.642720    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b2062b9-2ade-4b61-a486-53678a93fc79-extensions-socket-volume\") pod \"7b2062b9-2ade-4b61-a486-53678a93fc79\" (UID: \"7b2062b9-2ade-4b61-a486-53678a93fc79\") "
May 21 17:53:10.642925 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:10.642773    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flnqj\" (UniqueName: \"kubernetes.io/projected/7b2062b9-2ade-4b61-a486-53678a93fc79-kube-api-access-flnqj\") pod \"7b2062b9-2ade-4b61-a486-53678a93fc79\" (UID: \"7b2062b9-2ade-4b61-a486-53678a93fc79\") "
May 21 17:53:10.643208 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:10.643184    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b2062b9-2ade-4b61-a486-53678a93fc79-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7b2062b9-2ade-4b61-a486-53678a93fc79" (UID: "7b2062b9-2ade-4b61-a486-53678a93fc79"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 21 17:53:10.644858 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:10.644830    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b2062b9-2ade-4b61-a486-53678a93fc79-kube-api-access-flnqj" (OuterVolumeSpecName: "kube-api-access-flnqj") pod "7b2062b9-2ade-4b61-a486-53678a93fc79" (UID: "7b2062b9-2ade-4b61-a486-53678a93fc79"). InnerVolumeSpecName "kube-api-access-flnqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 17:53:10.743372 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:10.743335    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b2062b9-2ade-4b61-a486-53678a93fc79-extensions-socket-volume\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:53:10.743372 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:10.743368    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flnqj\" (UniqueName: \"kubernetes.io/projected/7b2062b9-2ade-4b61-a486-53678a93fc79-kube-api-access-flnqj\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:53:11.322497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:11.322460    2558 generic.go:358] "Generic (PLEG): container finished" podID="7b2062b9-2ade-4b61-a486-53678a93fc79" containerID="d7c2968383d624941d6b1df15ac6fe6bc8edd63902175d0908ab01ad81f8d2a5" exitCode=0
May 21 17:53:11.322864 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:11.322518    2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"
May 21 17:53:11.322864 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:11.322548    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w" event={"ID":"7b2062b9-2ade-4b61-a486-53678a93fc79","Type":"ContainerDied","Data":"d7c2968383d624941d6b1df15ac6fe6bc8edd63902175d0908ab01ad81f8d2a5"}
May 21 17:53:11.322864 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:11.322586    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w" event={"ID":"7b2062b9-2ade-4b61-a486-53678a93fc79","Type":"ContainerDied","Data":"7fc8d1d797298eeec9a8d74cf90e4a414b5e0874010f83f2d77fd54e66519491"}
May 21 17:53:11.322864 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:11.322602    2558 scope.go:117] "RemoveContainer" containerID="d7c2968383d624941d6b1df15ac6fe6bc8edd63902175d0908ab01ad81f8d2a5"
May 21 17:53:11.331019 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:11.330997    2558 scope.go:117] "RemoveContainer" containerID="d7c2968383d624941d6b1df15ac6fe6bc8edd63902175d0908ab01ad81f8d2a5"
May 21 17:53:11.331256 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:53:11.331241    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7c2968383d624941d6b1df15ac6fe6bc8edd63902175d0908ab01ad81f8d2a5\": container with ID starting with d7c2968383d624941d6b1df15ac6fe6bc8edd63902175d0908ab01ad81f8d2a5 not found: ID does not exist" containerID="d7c2968383d624941d6b1df15ac6fe6bc8edd63902175d0908ab01ad81f8d2a5"
May 21 17:53:11.331315 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:11.331263    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7c2968383d624941d6b1df15ac6fe6bc8edd63902175d0908ab01ad81f8d2a5"} err="failed to get container status \"d7c2968383d624941d6b1df15ac6fe6bc8edd63902175d0908ab01ad81f8d2a5\": rpc error: code = NotFound desc = could not find container \"d7c2968383d624941d6b1df15ac6fe6bc8edd63902175d0908ab01ad81f8d2a5\": container with ID starting with d7c2968383d624941d6b1df15ac6fe6bc8edd63902175d0908ab01ad81f8d2a5 not found: ID does not exist"
May 21 17:53:11.343735 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:11.343704    2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"]
May 21 17:53:11.346937 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:11.346916    2558 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-x2z2w"]
May 21 17:53:12.507014 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:12.506981    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b2062b9-2ade-4b61-a486-53678a93fc79" path="/var/lib/kubelet/pods/7b2062b9-2ade-4b61-a486-53678a93fc79/volumes"
May 21 17:53:14.557062 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.557027    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"]
May 21 17:53:14.557965 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.557937    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23e46708-7b6e-47ab-bfe5-569eec930cb3" containerName="manager"
May 21 17:53:14.557965 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.557965    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="23e46708-7b6e-47ab-bfe5-569eec930cb3" containerName="manager"
May 21 17:53:14.558167 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.557988    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b2062b9-2ade-4b61-a486-53678a93fc79" containerName="manager"
May 21 17:53:14.558167 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.557998    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b2062b9-2ade-4b61-a486-53678a93fc79" containerName="manager"
May 21 17:53:14.558167 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.558088    2558 memory_manager.go:356] "RemoveStaleState removing state" podUID="7b2062b9-2ade-4b61-a486-53678a93fc79" containerName="manager"
May 21 17:53:14.558167 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.558102    2558 memory_manager.go:356] "RemoveStaleState removing state" podUID="23e46708-7b6e-47ab-bfe5-569eec930cb3" containerName="manager"
May 21 17:53:14.562565 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.562541    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.564659 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.564637    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-5xt7p\""
May 21 17:53:14.571687 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.571385    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b36bcdb0-9066-4400-83d2-0bd313fa07e4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.571687 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.571446    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b36bcdb0-9066-4400-83d2-0bd313fa07e4-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.571687 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.571476    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b36bcdb0-9066-4400-83d2-0bd313fa07e4-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.571687 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.571516    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b36bcdb0-9066-4400-83d2-0bd313fa07e4-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.571687 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.571545    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b36bcdb0-9066-4400-83d2-0bd313fa07e4-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.571687 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.571599    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b36bcdb0-9066-4400-83d2-0bd313fa07e4-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.571687 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.571625    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b36bcdb0-9066-4400-83d2-0bd313fa07e4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.571687 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.571677    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b36bcdb0-9066-4400-83d2-0bd313fa07e4-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.572164 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.571724    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9x28\" (UniqueName: \"kubernetes.io/projected/b36bcdb0-9066-4400-83d2-0bd313fa07e4-kube-api-access-s9x28\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.572779 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.572302    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"]
May 21 17:53:14.672489 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.672453    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b36bcdb0-9066-4400-83d2-0bd313fa07e4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.672489 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.672494    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b36bcdb0-9066-4400-83d2-0bd313fa07e4-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.672739 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.672513    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b36bcdb0-9066-4400-83d2-0bd313fa07e4-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.672739 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.672537    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b36bcdb0-9066-4400-83d2-0bd313fa07e4-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.672739 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.672567    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b36bcdb0-9066-4400-83d2-0bd313fa07e4-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.672739 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.672604    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b36bcdb0-9066-4400-83d2-0bd313fa07e4-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.672739 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.672622    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b36bcdb0-9066-4400-83d2-0bd313fa07e4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.672739 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.672659    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b36bcdb0-9066-4400-83d2-0bd313fa07e4-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.672739 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.672692    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s9x28\" (UniqueName: \"kubernetes.io/projected/b36bcdb0-9066-4400-83d2-0bd313fa07e4-kube-api-access-s9x28\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.673150 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.673125    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b36bcdb0-9066-4400-83d2-0bd313fa07e4-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.673416 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.673387    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b36bcdb0-9066-4400-83d2-0bd313fa07e4-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.673553 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.673526    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b36bcdb0-9066-4400-83d2-0bd313fa07e4-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.673553 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.673489    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b36bcdb0-9066-4400-83d2-0bd313fa07e4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.673729 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.673604    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b36bcdb0-9066-4400-83d2-0bd313fa07e4-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.675165 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.675145    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b36bcdb0-9066-4400-83d2-0bd313fa07e4-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.675252 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.675199    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b36bcdb0-9066-4400-83d2-0bd313fa07e4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.679974 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.679954    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b36bcdb0-9066-4400-83d2-0bd313fa07e4-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.680089 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.679991    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9x28\" (UniqueName: \"kubernetes.io/projected/b36bcdb0-9066-4400-83d2-0bd313fa07e4-kube-api-access-s9x28\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-mnf2l\" (UID: \"b36bcdb0-9066-4400-83d2-0bd313fa07e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:14.877160 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:14.877074    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:15.016178 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:15.016125    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"]
May 21 17:53:15.022505 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:53:15.022462    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb36bcdb0_9066_4400_83d2_0bd313fa07e4.slice/crio-22cb0c9b8d8ffc0230d51409532acb41832a29723465c1bbf48c34da5d861c86 WatchSource:0}: Error finding container 22cb0c9b8d8ffc0230d51409532acb41832a29723465c1bbf48c34da5d861c86: Status 404 returned error can't find the container with id 22cb0c9b8d8ffc0230d51409532acb41832a29723465c1bbf48c34da5d861c86
May 21 17:53:15.338339 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:15.338301    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l" event={"ID":"b36bcdb0-9066-4400-83d2-0bd313fa07e4","Type":"ContainerStarted","Data":"22cb0c9b8d8ffc0230d51409532acb41832a29723465c1bbf48c34da5d861c86"}
May 21 17:53:18.917810 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:18.917754    2558 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
May 21 17:53:18.918054 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:18.917853    2558 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
May 21 17:53:18.918054 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:18.917885    2558 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"}
May 21 17:53:19.352552 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:19.352464    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l" event={"ID":"b36bcdb0-9066-4400-83d2-0bd313fa07e4","Type":"ContainerStarted","Data":"c1e1365e931571fa11c9285dcf3969456fdc66d59a819b45a740ac2b0da896bb"}
May 21 17:53:19.369411 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:19.369353    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l" podStartSLOduration=1.476415815 podStartE2EDuration="5.369338939s" podCreationTimestamp="2026-05-21 17:53:14 +0000 UTC" firstStartedPulling="2026-05-21 17:53:15.024571895 +0000 UTC m=+635.134964868" lastFinishedPulling="2026-05-21 17:53:18.91749502 +0000 UTC m=+639.027887992" observedRunningTime="2026-05-21 17:53:19.368584066 +0000 UTC m=+639.478977059" watchObservedRunningTime="2026-05-21 17:53:19.369338939 +0000 UTC m=+639.479731922"
May 21 17:53:19.878164 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:19.878129    2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:19.882767 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:19.882744    2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:20.355856 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:20.355826    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:20.356723 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:20.356704    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-mnf2l"
May 21 17:53:28.353266 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:28.353233    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-p84hb"]
May 21 17:53:28.356667 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:28.356645    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb"
May 21 17:53:28.358662 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:28.358640    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
May 21 17:53:28.358662 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:28.358656    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-97l5b\""
May 21 17:53:28.363728 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:28.363703    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-p84hb"]
May 21 17:53:28.460509 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:28.460475    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-p84hb"]
May 21 17:53:28.489256 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:28.489221    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/469161a8-d880-4455-8875-35e052bf2714-config-file\") pod \"limitador-limitador-7d549b5b-p84hb\" (UID: \"469161a8-d880-4455-8875-35e052bf2714\") " pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb"
May 21 17:53:28.489411 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:28.489274    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcfv5\" (UniqueName: \"kubernetes.io/projected/469161a8-d880-4455-8875-35e052bf2714-kube-api-access-fcfv5\") pod \"limitador-limitador-7d549b5b-p84hb\" (UID: \"469161a8-d880-4455-8875-35e052bf2714\") " pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb"
May 21 17:53:28.589757 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:28.589714    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/469161a8-d880-4455-8875-35e052bf2714-config-file\") pod \"limitador-limitador-7d549b5b-p84hb\" (UID: \"469161a8-d880-4455-8875-35e052bf2714\") " pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb"
May 21 17:53:28.589939 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:28.589774    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fcfv5\" (UniqueName: \"kubernetes.io/projected/469161a8-d880-4455-8875-35e052bf2714-kube-api-access-fcfv5\") pod \"limitador-limitador-7d549b5b-p84hb\" (UID: \"469161a8-d880-4455-8875-35e052bf2714\") " pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb"
May 21 17:53:28.590497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:28.590477    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/469161a8-d880-4455-8875-35e052bf2714-config-file\") pod \"limitador-limitador-7d549b5b-p84hb\" (UID: \"469161a8-d880-4455-8875-35e052bf2714\") " pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb"
May 21 17:53:28.596799 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:28.596764    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcfv5\" (UniqueName: \"kubernetes.io/projected/469161a8-d880-4455-8875-35e052bf2714-kube-api-access-fcfv5\") pod \"limitador-limitador-7d549b5b-p84hb\" (UID: \"469161a8-d880-4455-8875-35e052bf2714\") " pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb"
May 21 17:53:28.667824 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:28.667768    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb"
May 21 17:53:28.800729 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:28.800703    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-p84hb"]
May 21 17:53:28.803257 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:53:28.803229    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod469161a8_d880_4455_8875_35e052bf2714.slice/crio-0107ff388eb7e370d3e77018d89f23c77cc61df0746f597edf2e8779887d304b WatchSource:0}: Error finding container 0107ff388eb7e370d3e77018d89f23c77cc61df0746f597edf2e8779887d304b: Status 404 returned error can't find the container with id 0107ff388eb7e370d3e77018d89f23c77cc61df0746f597edf2e8779887d304b
May 21 17:53:29.387594 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:29.387557    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb" event={"ID":"469161a8-d880-4455-8875-35e052bf2714","Type":"ContainerStarted","Data":"0107ff388eb7e370d3e77018d89f23c77cc61df0746f597edf2e8779887d304b"}
May 21 17:53:32.398403 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:32.398365    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb" event={"ID":"469161a8-d880-4455-8875-35e052bf2714","Type":"ContainerStarted","Data":"eedd7fd3271451d73e811dab9c1bbc830e0ef772fb9e4e1077d4338bad4cc5bf"}
May 21 17:53:32.398843 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:32.398593    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb"
May 21 17:53:32.413889 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:32.413847    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb" podStartSLOduration=1.81413681 podStartE2EDuration="4.413835704s" podCreationTimestamp="2026-05-21 17:53:28 +0000 UTC" firstStartedPulling="2026-05-21 17:53:28.804962517 +0000 UTC m=+648.915355475" lastFinishedPulling="2026-05-21 17:53:31.404661411 +0000 UTC m=+651.515054369" observedRunningTime="2026-05-21 17:53:32.412072884 +0000 UTC m=+652.522465858" watchObservedRunningTime="2026-05-21 17:53:32.413835704 +0000 UTC m=+652.524228684"
May 21 17:53:43.403133 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:43.403097    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb"
May 21 17:53:43.586075 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:43.586039    2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-p84hb"]
May 21 17:53:43.586268 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:43.586245    2558 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb" podUID="469161a8-d880-4455-8875-35e052bf2714" containerName="limitador" containerID="cri-o://eedd7fd3271451d73e811dab9c1bbc830e0ef772fb9e4e1077d4338bad4cc5bf" gracePeriod=30
May 21 17:53:44.439988 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:44.439956    2558 generic.go:358] "Generic (PLEG): container finished" podID="469161a8-d880-4455-8875-35e052bf2714" containerID="eedd7fd3271451d73e811dab9c1bbc830e0ef772fb9e4e1077d4338bad4cc5bf" exitCode=0
May 21 17:53:44.440320 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:44.440002    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb" event={"ID":"469161a8-d880-4455-8875-35e052bf2714","Type":"ContainerDied","Data":"eedd7fd3271451d73e811dab9c1bbc830e0ef772fb9e4e1077d4338bad4cc5bf"}
May 21 17:53:44.524201 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:44.524178    2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb"
May 21 17:53:44.631113 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:44.631079    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcfv5\" (UniqueName: \"kubernetes.io/projected/469161a8-d880-4455-8875-35e052bf2714-kube-api-access-fcfv5\") pod \"469161a8-d880-4455-8875-35e052bf2714\" (UID: \"469161a8-d880-4455-8875-35e052bf2714\") "
May 21 17:53:44.631351 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:44.631119    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/469161a8-d880-4455-8875-35e052bf2714-config-file\") pod \"469161a8-d880-4455-8875-35e052bf2714\" (UID: \"469161a8-d880-4455-8875-35e052bf2714\") "
May 21 17:53:44.631528 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:44.631500    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/469161a8-d880-4455-8875-35e052bf2714-config-file" (OuterVolumeSpecName: "config-file") pod "469161a8-d880-4455-8875-35e052bf2714" (UID: "469161a8-d880-4455-8875-35e052bf2714"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 21 17:53:44.633168 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:44.633145    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/469161a8-d880-4455-8875-35e052bf2714-kube-api-access-fcfv5" (OuterVolumeSpecName: "kube-api-access-fcfv5") pod "469161a8-d880-4455-8875-35e052bf2714" (UID: "469161a8-d880-4455-8875-35e052bf2714"). InnerVolumeSpecName "kube-api-access-fcfv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 17:53:44.731832 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:44.731778    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcfv5\" (UniqueName: \"kubernetes.io/projected/469161a8-d880-4455-8875-35e052bf2714-kube-api-access-fcfv5\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:53:44.731832 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:44.731828    2558 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/469161a8-d880-4455-8875-35e052bf2714-config-file\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:53:45.444179 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:45.444137    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb" event={"ID":"469161a8-d880-4455-8875-35e052bf2714","Type":"ContainerDied","Data":"0107ff388eb7e370d3e77018d89f23c77cc61df0746f597edf2e8779887d304b"}
May 21 17:53:45.444179 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:45.444171    2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-p84hb"
May 21 17:53:45.444179 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:45.444187    2558 scope.go:117] "RemoveContainer" containerID="eedd7fd3271451d73e811dab9c1bbc830e0ef772fb9e4e1077d4338bad4cc5bf"
May 21 17:53:45.464394 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:45.464373    2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-p84hb"]
May 21 17:53:45.468226 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:45.468201    2558 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-p84hb"]
May 21 17:53:46.278341 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.278299    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-zfcb6"]
May 21 17:53:46.278633 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.278621    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="469161a8-d880-4455-8875-35e052bf2714" containerName="limitador"
May 21 17:53:46.278679 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.278635    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="469161a8-d880-4455-8875-35e052bf2714" containerName="limitador"
May 21 17:53:46.278713 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.278681    2558 memory_manager.go:356] "RemoveStaleState removing state" podUID="469161a8-d880-4455-8875-35e052bf2714" containerName="limitador"
May 21 17:53:46.282991 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.282971    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-zfcb6"
May 21 17:53:46.285115 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.285093    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
May 21 17:53:46.285255 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.285128    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-bq2ng\""
May 21 17:53:46.289834 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.289811    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-zfcb6"]
May 21 17:53:46.445391 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.445365    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7h8n\" (UniqueName: \"kubernetes.io/projected/608888ad-feb3-4a72-a4cb-651217086174-kube-api-access-r7h8n\") pod \"postgres-68c4fbbd6f-zfcb6\" (UID: \"608888ad-feb3-4a72-a4cb-651217086174\") " pod="opendatahub/postgres-68c4fbbd6f-zfcb6"
May 21 17:53:46.445772 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.445405    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/608888ad-feb3-4a72-a4cb-651217086174-data\") pod \"postgres-68c4fbbd6f-zfcb6\" (UID: \"608888ad-feb3-4a72-a4cb-651217086174\") " pod="opendatahub/postgres-68c4fbbd6f-zfcb6"
May 21 17:53:46.507510 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.507473    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="469161a8-d880-4455-8875-35e052bf2714" path="/var/lib/kubelet/pods/469161a8-d880-4455-8875-35e052bf2714/volumes"
May 21 17:53:46.546159 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.546083    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r7h8n\" (UniqueName: \"kubernetes.io/projected/608888ad-feb3-4a72-a4cb-651217086174-kube-api-access-r7h8n\") pod \"postgres-68c4fbbd6f-zfcb6\" (UID: \"608888ad-feb3-4a72-a4cb-651217086174\") " pod="opendatahub/postgres-68c4fbbd6f-zfcb6"
May 21 17:53:46.546159 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.546124    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/608888ad-feb3-4a72-a4cb-651217086174-data\") pod \"postgres-68c4fbbd6f-zfcb6\" (UID: \"608888ad-feb3-4a72-a4cb-651217086174\") " pod="opendatahub/postgres-68c4fbbd6f-zfcb6"
May 21 17:53:46.546484 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.546469    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/608888ad-feb3-4a72-a4cb-651217086174-data\") pod \"postgres-68c4fbbd6f-zfcb6\" (UID: \"608888ad-feb3-4a72-a4cb-651217086174\") " pod="opendatahub/postgres-68c4fbbd6f-zfcb6"
May 21 17:53:46.553240 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.553222    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7h8n\" (UniqueName: \"kubernetes.io/projected/608888ad-feb3-4a72-a4cb-651217086174-kube-api-access-r7h8n\") pod \"postgres-68c4fbbd6f-zfcb6\" (UID: \"608888ad-feb3-4a72-a4cb-651217086174\") " pod="opendatahub/postgres-68c4fbbd6f-zfcb6"
May 21 17:53:46.602229 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.602202    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-zfcb6"
May 21 17:53:46.724529 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:46.724499    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-zfcb6"]
May 21 17:53:46.727585 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:53:46.727555    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod608888ad_feb3_4a72_a4cb_651217086174.slice/crio-f696e26fa37b610d6a41f4c6faed3250efa1338a37bd9a6ba051bfc35cb61c75 WatchSource:0}: Error finding container f696e26fa37b610d6a41f4c6faed3250efa1338a37bd9a6ba051bfc35cb61c75: Status 404 returned error can't find the container with id f696e26fa37b610d6a41f4c6faed3250efa1338a37bd9a6ba051bfc35cb61c75
May 21 17:53:47.453470 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:47.453432    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-zfcb6" event={"ID":"608888ad-feb3-4a72-a4cb-651217086174","Type":"ContainerStarted","Data":"f696e26fa37b610d6a41f4c6faed3250efa1338a37bd9a6ba051bfc35cb61c75"}
May 21 17:53:52.470768 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:52.470733    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-zfcb6" event={"ID":"608888ad-feb3-4a72-a4cb-651217086174","Type":"ContainerStarted","Data":"bf08b3d24d3cedf1fdbcfd6ccd4621a33fa89ba14f3339aa29f0883023569211"}
May 21 17:53:52.471144 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:52.470868    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-zfcb6"
May 21 17:53:52.483605 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:52.483556    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-zfcb6" podStartSLOduration=1.677621254 podStartE2EDuration="6.483541956s" podCreationTimestamp="2026-05-21 17:53:46 +0000 UTC" firstStartedPulling="2026-05-21 17:53:46.728778451 +0000 UTC m=+666.839171413" lastFinishedPulling="2026-05-21 17:53:51.534699146 +0000 UTC m=+671.645092115" observedRunningTime="2026-05-21 17:53:52.483085409 +0000 UTC m=+672.593478390" watchObservedRunningTime="2026-05-21 17:53:52.483541956 +0000 UTC m=+672.593934937"
May 21 17:53:58.501887 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:53:58.501850    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-zfcb6"
May 21 17:54:04.868350 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:04.868315    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d595d9556-4tj5n"]
May 21 17:54:04.876610 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:04.876584    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d595d9556-4tj5n"
May 21 17:54:04.878808 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:04.878771    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-h6z9t\""
May 21 17:54:04.878943 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:04.878771    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
May 21 17:54:04.879635 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:04.879611    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d595d9556-4tj5n"]
May 21 17:54:04.886590 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:04.886569    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph66w\" (UniqueName: \"kubernetes.io/projected/3e180fa0-17ce-4b54-8255-8002c6c9716e-kube-api-access-ph66w\") pod \"maas-controller-6d595d9556-4tj5n\" (UID: \"3e180fa0-17ce-4b54-8255-8002c6c9716e\") " pod="opendatahub/maas-controller-6d595d9556-4tj5n"
May 21 17:54:04.982883 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:04.982847    2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d595d9556-4tj5n"]
May 21 17:54:04.983106 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:54:04.983085    2558 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-ph66w], unattached volumes=[], failed to process volumes=[]: context canceled" pod="opendatahub/maas-controller-6d595d9556-4tj5n" podUID="3e180fa0-17ce-4b54-8255-8002c6c9716e"
May 21 17:54:04.987438 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:04.987407    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ph66w\" (UniqueName: \"kubernetes.io/projected/3e180fa0-17ce-4b54-8255-8002c6c9716e-kube-api-access-ph66w\") pod \"maas-controller-6d595d9556-4tj5n\" (UID: \"3e180fa0-17ce-4b54-8255-8002c6c9716e\") " pod="opendatahub/maas-controller-6d595d9556-4tj5n"
May 21 17:54:04.996232 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:04.996209    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph66w\" (UniqueName: \"kubernetes.io/projected/3e180fa0-17ce-4b54-8255-8002c6c9716e-kube-api-access-ph66w\") pod \"maas-controller-6d595d9556-4tj5n\" (UID: \"3e180fa0-17ce-4b54-8255-8002c6c9716e\") " pod="opendatahub/maas-controller-6d595d9556-4tj5n"
May 21 17:54:05.510262 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:05.510232    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d595d9556-4tj5n"
May 21 17:54:05.514924 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:05.514903    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d595d9556-4tj5n"
May 21 17:54:05.592125 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:05.592091    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ph66w\" (UniqueName: \"kubernetes.io/projected/3e180fa0-17ce-4b54-8255-8002c6c9716e-kube-api-access-ph66w\") pod \"3e180fa0-17ce-4b54-8255-8002c6c9716e\" (UID: \"3e180fa0-17ce-4b54-8255-8002c6c9716e\") "
May 21 17:54:05.594134 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:05.594099    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e180fa0-17ce-4b54-8255-8002c6c9716e-kube-api-access-ph66w" (OuterVolumeSpecName: "kube-api-access-ph66w") pod "3e180fa0-17ce-4b54-8255-8002c6c9716e" (UID: "3e180fa0-17ce-4b54-8255-8002c6c9716e"). InnerVolumeSpecName "kube-api-access-ph66w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 17:54:05.693500 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:05.693466    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ph66w\" (UniqueName: \"kubernetes.io/projected/3e180fa0-17ce-4b54-8255-8002c6c9716e-kube-api-access-ph66w\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 17:54:06.513456 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:06.513429    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d595d9556-4tj5n"
May 21 17:54:06.536506 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:06.536475    2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d595d9556-4tj5n"]
May 21 17:54:06.540415 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:06.540393    2558 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6d595d9556-4tj5n"]
May 21 17:54:08.507337 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:08.507303    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e180fa0-17ce-4b54-8255-8002c6c9716e" path="/var/lib/kubelet/pods/3e180fa0-17ce-4b54-8255-8002c6c9716e/volumes"
May 21 17:54:51.292353 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:51.292321    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-84c48d449b-q96vd"]
May 21 17:54:51.296739 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:51.296723    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-84c48d449b-q96vd"
May 21 17:54:51.298621 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:51.298603    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
May 21 17:54:51.298721 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:51.298620    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
May 21 17:54:51.298721 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:51.298626    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-trsvf\""
May 21 17:54:51.305949 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:51.305925    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-84c48d449b-q96vd"]
May 21 17:54:51.477122 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:51.477087    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz9tv\" (UniqueName: \"kubernetes.io/projected/bc0d0284-fd95-4b5e-8986-a24e59c951e0-kube-api-access-hz9tv\") pod \"maas-api-84c48d449b-q96vd\" (UID: \"bc0d0284-fd95-4b5e-8986-a24e59c951e0\") " pod="opendatahub/maas-api-84c48d449b-q96vd"
May 21 17:54:51.477292 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:51.477135    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc0d0284-fd95-4b5e-8986-a24e59c951e0-maas-api-tls\") pod \"maas-api-84c48d449b-q96vd\" (UID: \"bc0d0284-fd95-4b5e-8986-a24e59c951e0\") " pod="opendatahub/maas-api-84c48d449b-q96vd"
May 21 17:54:51.578349 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:51.578273    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hz9tv\" (UniqueName: \"kubernetes.io/projected/bc0d0284-fd95-4b5e-8986-a24e59c951e0-kube-api-access-hz9tv\") pod \"maas-api-84c48d449b-q96vd\" (UID: \"bc0d0284-fd95-4b5e-8986-a24e59c951e0\") " pod="opendatahub/maas-api-84c48d449b-q96vd"
May 21 17:54:51.578349 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:51.578315    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc0d0284-fd95-4b5e-8986-a24e59c951e0-maas-api-tls\") pod \"maas-api-84c48d449b-q96vd\" (UID: \"bc0d0284-fd95-4b5e-8986-a24e59c951e0\") " pod="opendatahub/maas-api-84c48d449b-q96vd"
May 21 17:54:51.578518 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:54:51.578427    2558 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found
May 21 17:54:51.578518 ip-10-0-135-125 kubenswrapper[2558]: E0521 17:54:51.578479    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc0d0284-fd95-4b5e-8986-a24e59c951e0-maas-api-tls podName:bc0d0284-fd95-4b5e-8986-a24e59c951e0 nodeName:}" failed. No retries permitted until 2026-05-21 17:54:52.078464421 +0000 UTC m=+732.188857383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bc0d0284-fd95-4b5e-8986-a24e59c951e0-maas-api-tls") pod "maas-api-84c48d449b-q96vd" (UID: "bc0d0284-fd95-4b5e-8986-a24e59c951e0") : secret "maas-api-serving-cert" not found
May 21 17:54:51.594272 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:51.594244    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz9tv\" (UniqueName: \"kubernetes.io/projected/bc0d0284-fd95-4b5e-8986-a24e59c951e0-kube-api-access-hz9tv\") pod \"maas-api-84c48d449b-q96vd\" (UID: \"bc0d0284-fd95-4b5e-8986-a24e59c951e0\") " pod="opendatahub/maas-api-84c48d449b-q96vd"
May 21 17:54:52.082350 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:52.082306    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc0d0284-fd95-4b5e-8986-a24e59c951e0-maas-api-tls\") pod \"maas-api-84c48d449b-q96vd\" (UID: \"bc0d0284-fd95-4b5e-8986-a24e59c951e0\") " pod="opendatahub/maas-api-84c48d449b-q96vd"
May 21 17:54:52.084637 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:52.084611    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc0d0284-fd95-4b5e-8986-a24e59c951e0-maas-api-tls\") pod \"maas-api-84c48d449b-q96vd\" (UID: \"bc0d0284-fd95-4b5e-8986-a24e59c951e0\") " pod="opendatahub/maas-api-84c48d449b-q96vd"
May 21 17:54:52.207111 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:52.207072    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-84c48d449b-q96vd"
May 21 17:54:52.341584 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:52.341514    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-84c48d449b-q96vd"]
May 21 17:54:52.344857 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:54:52.344827    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc0d0284_fd95_4b5e_8986_a24e59c951e0.slice/crio-a668b1fa2181eabb8298aca536648246bdc18b55fde6b6377628fd66f8c0ddd1 WatchSource:0}: Error finding container a668b1fa2181eabb8298aca536648246bdc18b55fde6b6377628fd66f8c0ddd1: Status 404 returned error can't find the container with id a668b1fa2181eabb8298aca536648246bdc18b55fde6b6377628fd66f8c0ddd1
May 21 17:54:52.653958 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:52.653865    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-84c48d449b-q96vd" event={"ID":"bc0d0284-fd95-4b5e-8986-a24e59c951e0","Type":"ContainerStarted","Data":"a668b1fa2181eabb8298aca536648246bdc18b55fde6b6377628fd66f8c0ddd1"}
May 21 17:54:55.667595 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:55.667562    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-84c48d449b-q96vd" event={"ID":"bc0d0284-fd95-4b5e-8986-a24e59c951e0","Type":"ContainerStarted","Data":"9e7fd36d612ad9b048830b2a336b5cda17710fafa1435653016ff19db873f1d7"}
May 21 17:54:55.668011 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:55.667761    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-84c48d449b-q96vd"
May 21 17:54:55.682013 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:54:55.681971    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-84c48d449b-q96vd" podStartSLOduration=1.998535216 podStartE2EDuration="4.681959113s" podCreationTimestamp="2026-05-21 17:54:51 +0000 UTC" firstStartedPulling="2026-05-21 17:54:52.346533268 +0000 UTC m=+732.456926225" lastFinishedPulling="2026-05-21 17:54:55.029957159 +0000 UTC m=+735.140350122" observedRunningTime="2026-05-21 17:54:55.680926209 +0000 UTC m=+735.791319189" watchObservedRunningTime="2026-05-21 17:54:55.681959113 +0000 UTC m=+735.792352093"
May 21 17:55:01.675909 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:01.675879    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-84c48d449b-q96vd"
May 21 17:55:25.148728 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.148696    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"]
May 21 17:55:25.152426 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.152402    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.155073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.155052    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
May 21 17:55:25.155073 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.155063    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
May 21 17:55:25.155219 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.155070    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bqdm5\""
May 21 17:55:25.155219 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.155131    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
May 21 17:55:25.161903 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.161882    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"]
May 21 17:55:25.257599 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.257567    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.257825 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.257610    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk7gl\" (UniqueName: \"kubernetes.io/projected/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-kube-api-access-mk7gl\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.257825 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.257660    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.257825 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.257686    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.257825 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.257811    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.258053 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.257861    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.359055 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.359014    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.359055 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.359060    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.359293 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.359107    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.359293 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.359123    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mk7gl\" (UniqueName: \"kubernetes.io/projected/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-kube-api-access-mk7gl\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.359293 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.359150    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.359293 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.359184    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.360285 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.359603    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.360285 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.359636    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.360285 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.359816    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.361435 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.361415    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.361699 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.361681    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.367560 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.367536    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk7gl\" (UniqueName: \"kubernetes.io/projected/84cc45f8-0474-479a-bf08-b8b9fe1bbe8d-kube-api-access-mk7gl\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4\" (UID: \"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.462956 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.462909    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:25.593466 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.593438    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"]
May 21 17:55:25.596287 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:55:25.596262    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84cc45f8_0474_479a_bf08_b8b9fe1bbe8d.slice/crio-7f5c556a96749fe20451dfe623cf1560a01661c0bcd235ec3096d9a9207f06b2 WatchSource:0}: Error finding container 7f5c556a96749fe20451dfe623cf1560a01661c0bcd235ec3096d9a9207f06b2: Status 404 returned error can't find the container with id 7f5c556a96749fe20451dfe623cf1560a01661c0bcd235ec3096d9a9207f06b2
May 21 17:55:25.598512 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.598487    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 21 17:55:25.768051 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:25.767969    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4" event={"ID":"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d","Type":"ContainerStarted","Data":"7f5c556a96749fe20451dfe623cf1560a01661c0bcd235ec3096d9a9207f06b2"}
May 21 17:55:31.794882 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:31.794837    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4" event={"ID":"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d","Type":"ContainerStarted","Data":"618a5498b22e8458262ae1a7053431d80d9e9777e9fa457599a49424ff66d98d"}
May 21 17:55:36.814233 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:36.814196    2558 generic.go:358] "Generic (PLEG): container finished" podID="84cc45f8-0474-479a-bf08-b8b9fe1bbe8d" containerID="618a5498b22e8458262ae1a7053431d80d9e9777e9fa457599a49424ff66d98d" exitCode=0
May 21 17:55:36.814645 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:36.814332    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4" event={"ID":"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d","Type":"ContainerDied","Data":"618a5498b22e8458262ae1a7053431d80d9e9777e9fa457599a49424ff66d98d"}
May 21 17:55:37.349884 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.349731    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"]
May 21 17:55:37.354677 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.354654    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.356876 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.356854    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
May 21 17:55:37.366670 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.366626    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"]
May 21 17:55:37.465378 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.465235    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.465378 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.465292    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.465378 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.465331    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpqpq\" (UniqueName: \"kubernetes.io/projected/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-kube-api-access-tpqpq\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.465632 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.465402    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.465632 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.465454    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.465632 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.465519    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.566745 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.566705    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.566911 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.566762    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.566911 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.566820    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tpqpq\" (UniqueName: \"kubernetes.io/projected/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-kube-api-access-tpqpq\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.566911 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.566876    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.567016 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.566920    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.567016 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.566985    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.567417 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.567369    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.567417 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.567401    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.567864 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.567838    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.569699 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.569644    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.569874 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.569856    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.574273 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.574228    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpqpq\" (UniqueName: \"kubernetes.io/projected/4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a-kube-api-access-tpqpq\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-bcn2q\" (UID: \"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:37.669733 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:37.669696    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:38.124507 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:38.124478    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"]
May 21 17:55:38.129055 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:55:38.129032    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc851e8_0c60_4e3d_b5f2_ed2d6368d60a.slice/crio-945c16bb0b76bd4423b99759423d325decef1313808e8a8702cecffc3f97cb46 WatchSource:0}: Error finding container 945c16bb0b76bd4423b99759423d325decef1313808e8a8702cecffc3f97cb46: Status 404 returned error can't find the container with id 945c16bb0b76bd4423b99759423d325decef1313808e8a8702cecffc3f97cb46
May 21 17:55:38.824079 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:38.824042    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4" event={"ID":"84cc45f8-0474-479a-bf08-b8b9fe1bbe8d","Type":"ContainerStarted","Data":"ec18994e7da8f40cfc9e8993e038970fc1eb1ccbbad11659cfef970d8f913ccd"}
May 21 17:55:38.824321 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:38.824282    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:38.825767 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:38.825744    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q" event={"ID":"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a","Type":"ContainerStarted","Data":"8a74932a49b5eef0985c277afc297226cd7b9b9e2df1481bbe03ed5fb0c68f77"}
May 21 17:55:38.825897 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:38.825773    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q" event={"ID":"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a","Type":"ContainerStarted","Data":"945c16bb0b76bd4423b99759423d325decef1313808e8a8702cecffc3f97cb46"}
May 21 17:55:38.841358 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:38.841299    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4" podStartSLOduration=1.390547698 podStartE2EDuration="13.841282058s" podCreationTimestamp="2026-05-21 17:55:25 +0000 UTC" firstStartedPulling="2026-05-21 17:55:25.59867686 +0000 UTC m=+765.709069832" lastFinishedPulling="2026-05-21 17:55:38.049411229 +0000 UTC m=+778.159804192" observedRunningTime="2026-05-21 17:55:38.839904381 +0000 UTC m=+778.950297363" watchObservedRunningTime="2026-05-21 17:55:38.841282058 +0000 UTC m=+778.951675041"
May 21 17:55:43.844037 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:43.844003    2558 generic.go:358] "Generic (PLEG): container finished" podID="4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a" containerID="8a74932a49b5eef0985c277afc297226cd7b9b9e2df1481bbe03ed5fb0c68f77" exitCode=0
May 21 17:55:43.844409 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:43.844074    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q" event={"ID":"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a","Type":"ContainerDied","Data":"8a74932a49b5eef0985c277afc297226cd7b9b9e2df1481bbe03ed5fb0c68f77"}
May 21 17:55:44.849141 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:44.849105    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q" event={"ID":"4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a","Type":"ContainerStarted","Data":"10b8ac4f37e1c83564cc599dacea53be36d5dbf171de6e1b763eaf6acb243d11"}
May 21 17:55:44.849528 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:44.849327    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:44.866424 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:44.866378    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q" podStartSLOduration=7.60680538 podStartE2EDuration="7.86636486s" podCreationTimestamp="2026-05-21 17:55:37 +0000 UTC" firstStartedPulling="2026-05-21 17:55:43.844718914 +0000 UTC m=+783.955111873" lastFinishedPulling="2026-05-21 17:55:44.104278392 +0000 UTC m=+784.214671353" observedRunningTime="2026-05-21 17:55:44.865019354 +0000 UTC m=+784.975412334" watchObservedRunningTime="2026-05-21 17:55:44.86636486 +0000 UTC m=+784.976757840"
May 21 17:55:49.843024 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:49.842984    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4"
May 21 17:55:53.189294 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.189258    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"]
May 21 17:55:53.196315 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.196290    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.200617 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.200595    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
May 21 17:55:53.223036 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.223010    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"]
May 21 17:55:53.308918 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.308883    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.309092 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.308954    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.309092 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.308991    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.309092 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.309013    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrfdd\" (UniqueName: \"kubernetes.io/projected/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-kube-api-access-jrfdd\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.309092 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.309043    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.309092 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.309074    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.409871 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.409833    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.410037 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.409899    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.410037 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.409947    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.410037 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.409980    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jrfdd\" (UniqueName: \"kubernetes.io/projected/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-kube-api-access-jrfdd\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.410037 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.410011    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.410037 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.410034    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.410336 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.410301    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.410397 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.410367    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.410397 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.410375    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.414305 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.414282    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.419703 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.419656    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrfdd\" (UniqueName: \"kubernetes.io/projected/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-kube-api-access-jrfdd\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.419991 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.419969    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv\" (UID: \"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.507009 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.506925    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:55:53.643610 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.643585    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"]
May 21 17:55:53.646233 ip-10-0-135-125 kubenswrapper[2558]: W0521 17:55:53.646202    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f3bc2a1_79cc_42f8_9f4c_5536bf1f59cf.slice/crio-08154ea42c3bb8b37e91df6612c136c299ce41f77fb37b95e13c4c4f53ddbd88 WatchSource:0}: Error finding container 08154ea42c3bb8b37e91df6612c136c299ce41f77fb37b95e13c4c4f53ddbd88: Status 404 returned error can't find the container with id 08154ea42c3bb8b37e91df6612c136c299ce41f77fb37b95e13c4c4f53ddbd88
May 21 17:55:53.885175 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.885088    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv" event={"ID":"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf","Type":"ContainerStarted","Data":"f0b5375aec17f699121bc810e5b934d41f8ffe932ecead0b9d3367b3afe4e333"}
May 21 17:55:53.885175 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:53.885128    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv" event={"ID":"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf","Type":"ContainerStarted","Data":"08154ea42c3bb8b37e91df6612c136c299ce41f77fb37b95e13c4c4f53ddbd88"}
May 21 17:55:55.866746 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:55.866719    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-bcn2q"
May 21 17:55:59.907108 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:59.907072    2558 generic.go:358] "Generic (PLEG): container finished" podID="5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf" containerID="f0b5375aec17f699121bc810e5b934d41f8ffe932ecead0b9d3367b3afe4e333" exitCode=0
May 21 17:55:59.907497 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:55:59.907122    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv" event={"ID":"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf","Type":"ContainerDied","Data":"f0b5375aec17f699121bc810e5b934d41f8ffe932ecead0b9d3367b3afe4e333"}
May 21 17:56:00.911632 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:56:00.911601    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv" event={"ID":"5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf","Type":"ContainerStarted","Data":"a0f6d97345497a72e1d3410482a2a49b9fc8900170a9cb7e2dfeac9a46ce6e8d"}
May 21 17:56:00.912088 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:56:00.911858    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:56:00.929950 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:56:00.929908    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv" podStartSLOduration=7.482283922 podStartE2EDuration="7.92989421s" podCreationTimestamp="2026-05-21 17:55:53 +0000 UTC" firstStartedPulling="2026-05-21 17:55:59.907819895 +0000 UTC m=+800.018212854" lastFinishedPulling="2026-05-21 17:56:00.355430184 +0000 UTC m=+800.465823142" observedRunningTime="2026-05-21 17:56:00.927768266 +0000 UTC m=+801.038161248" watchObservedRunningTime="2026-05-21 17:56:00.92989421 +0000 UTC m=+801.040287234"
May 21 17:56:11.930554 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:56:11.930522    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv"
May 21 17:57:40.438146 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:57:40.438075    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 17:57:40.440200 ip-10-0-135-125 kubenswrapper[2558]: I0521 17:57:40.440167    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 18:02:40.461229 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:02:40.461200    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 18:02:40.463531 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:02:40.463507    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 18:07:40.483067 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:07:40.483036    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 18:07:40.485901 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:07:40.485880    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 18:08:32.327641 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:32.327599    2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"]
May 21 18:08:32.328198 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:32.327912    2558 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd" podUID="c29ae5f9-c77b-4c59-8afb-2b38d6be71cb" containerName="manager" containerID="cri-o://ad102817c5dab5046f9e67cb8e7dd4e1557ffaf11245b30edcc528332f422320" gracePeriod=10
May 21 18:08:33.062866 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:33.062844    2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"
May 21 18:08:33.158768 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:33.158738    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4bv7\" (UniqueName: \"kubernetes.io/projected/c29ae5f9-c77b-4c59-8afb-2b38d6be71cb-kube-api-access-f4bv7\") pod \"c29ae5f9-c77b-4c59-8afb-2b38d6be71cb\" (UID: \"c29ae5f9-c77b-4c59-8afb-2b38d6be71cb\") "
May 21 18:08:33.158948 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:33.158822    2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c29ae5f9-c77b-4c59-8afb-2b38d6be71cb-extensions-socket-volume\") pod \"c29ae5f9-c77b-4c59-8afb-2b38d6be71cb\" (UID: \"c29ae5f9-c77b-4c59-8afb-2b38d6be71cb\") "
May 21 18:08:33.159192 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:33.159160    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c29ae5f9-c77b-4c59-8afb-2b38d6be71cb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c29ae5f9-c77b-4c59-8afb-2b38d6be71cb" (UID: "c29ae5f9-c77b-4c59-8afb-2b38d6be71cb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 21 18:08:33.160688 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:33.160667    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c29ae5f9-c77b-4c59-8afb-2b38d6be71cb-kube-api-access-f4bv7" (OuterVolumeSpecName: "kube-api-access-f4bv7") pod "c29ae5f9-c77b-4c59-8afb-2b38d6be71cb" (UID: "c29ae5f9-c77b-4c59-8afb-2b38d6be71cb"). InnerVolumeSpecName "kube-api-access-f4bv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 21 18:08:33.260015 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:33.259928    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c29ae5f9-c77b-4c59-8afb-2b38d6be71cb-extensions-socket-volume\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 18:08:33.260015 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:33.259958    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4bv7\" (UniqueName: \"kubernetes.io/projected/c29ae5f9-c77b-4c59-8afb-2b38d6be71cb-kube-api-access-f4bv7\") on node \"ip-10-0-135-125.ec2.internal\" DevicePath \"\""
May 21 18:08:33.472490 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:33.472450    2558 generic.go:358] "Generic (PLEG): container finished" podID="c29ae5f9-c77b-4c59-8afb-2b38d6be71cb" containerID="ad102817c5dab5046f9e67cb8e7dd4e1557ffaf11245b30edcc528332f422320" exitCode=0
May 21 18:08:33.472902 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:33.472536    2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"
May 21 18:08:33.472902 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:33.472527    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd" event={"ID":"c29ae5f9-c77b-4c59-8afb-2b38d6be71cb","Type":"ContainerDied","Data":"ad102817c5dab5046f9e67cb8e7dd4e1557ffaf11245b30edcc528332f422320"}
May 21 18:08:33.472902 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:33.472640    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd" event={"ID":"c29ae5f9-c77b-4c59-8afb-2b38d6be71cb","Type":"ContainerDied","Data":"ae118587732edb98ea6c009324cabee53fa7a85afb4227f8ed44355335547c5d"}
May 21 18:08:33.472902 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:33.472662    2558 scope.go:117] "RemoveContainer" containerID="ad102817c5dab5046f9e67cb8e7dd4e1557ffaf11245b30edcc528332f422320"
May 21 18:08:33.481354 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:33.481334    2558 scope.go:117] "RemoveContainer" containerID="ad102817c5dab5046f9e67cb8e7dd4e1557ffaf11245b30edcc528332f422320"
May 21 18:08:33.481672 ip-10-0-135-125 kubenswrapper[2558]: E0521 18:08:33.481628    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad102817c5dab5046f9e67cb8e7dd4e1557ffaf11245b30edcc528332f422320\": container with ID starting with ad102817c5dab5046f9e67cb8e7dd4e1557ffaf11245b30edcc528332f422320 not found: ID does not exist" containerID="ad102817c5dab5046f9e67cb8e7dd4e1557ffaf11245b30edcc528332f422320"
May 21 18:08:33.481800 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:33.481678    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad102817c5dab5046f9e67cb8e7dd4e1557ffaf11245b30edcc528332f422320"} err="failed to get container status \"ad102817c5dab5046f9e67cb8e7dd4e1557ffaf11245b30edcc528332f422320\": rpc error: code = NotFound desc = could not find container \"ad102817c5dab5046f9e67cb8e7dd4e1557ffaf11245b30edcc528332f422320\": container with ID starting with ad102817c5dab5046f9e67cb8e7dd4e1557ffaf11245b30edcc528332f422320 not found: ID does not exist"
May 21 18:08:33.493929 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:33.493903    2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"]
May 21 18:08:33.497270 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:33.497239    2558 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-9glsd"]
May 21 18:08:34.506917 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:08:34.506889    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c29ae5f9-c77b-4c59-8afb-2b38d6be71cb" path="/var/lib/kubelet/pods/c29ae5f9-c77b-4c59-8afb-2b38d6be71cb/volumes"
May 21 18:09:38.476382 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.476232    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5dg5c"]
May 21 18:09:38.476946 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.476828    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c29ae5f9-c77b-4c59-8afb-2b38d6be71cb" containerName="manager"
May 21 18:09:38.476946 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.476849    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c29ae5f9-c77b-4c59-8afb-2b38d6be71cb" containerName="manager"
May 21 18:09:38.477071 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.476948    2558 memory_manager.go:356] "RemoveStaleState removing state" podUID="c29ae5f9-c77b-4c59-8afb-2b38d6be71cb" containerName="manager"
May 21 18:09:38.479844 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.479823    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5dg5c"
May 21 18:09:38.482778 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.482758    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
May 21 18:09:38.483153 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.483134    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6psh9\""
May 21 18:09:38.483412 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.483397    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
May 21 18:09:38.491181 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.491158    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5dg5c"]
May 21 18:09:38.522527 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.522493    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56f7e3f7-2529-4d95-9098-668b71ed866e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-5dg5c\" (UID: \"56f7e3f7-2529-4d95-9098-668b71ed866e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5dg5c"
May 21 18:09:38.522527 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.522530    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn29l\" (UniqueName: \"kubernetes.io/projected/56f7e3f7-2529-4d95-9098-668b71ed866e-kube-api-access-zn29l\") pod \"kuadrant-operator-controller-manager-55c7f4c975-5dg5c\" (UID: \"56f7e3f7-2529-4d95-9098-668b71ed866e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5dg5c"
May 21 18:09:38.623206 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.623163    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56f7e3f7-2529-4d95-9098-668b71ed866e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-5dg5c\" (UID: \"56f7e3f7-2529-4d95-9098-668b71ed866e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5dg5c"
May 21 18:09:38.623206 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.623207    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zn29l\" (UniqueName: \"kubernetes.io/projected/56f7e3f7-2529-4d95-9098-668b71ed866e-kube-api-access-zn29l\") pod \"kuadrant-operator-controller-manager-55c7f4c975-5dg5c\" (UID: \"56f7e3f7-2529-4d95-9098-668b71ed866e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5dg5c"
May 21 18:09:38.623672 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.623564    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56f7e3f7-2529-4d95-9098-668b71ed866e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-5dg5c\" (UID: \"56f7e3f7-2529-4d95-9098-668b71ed866e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5dg5c"
May 21 18:09:38.636215 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.636189    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn29l\" (UniqueName: \"kubernetes.io/projected/56f7e3f7-2529-4d95-9098-668b71ed866e-kube-api-access-zn29l\") pod \"kuadrant-operator-controller-manager-55c7f4c975-5dg5c\" (UID: \"56f7e3f7-2529-4d95-9098-668b71ed866e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5dg5c"
May 21 18:09:38.790483 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.790402    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5dg5c"
May 21 18:09:38.942395 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.942364    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5dg5c"]
May 21 18:09:38.944645 ip-10-0-135-125 kubenswrapper[2558]: W0521 18:09:38.944614    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56f7e3f7_2529_4d95_9098_668b71ed866e.slice/crio-d9d587fd92f25bb7dbd41df300f913a9cc49c39677c7217a4fbc014ebfb600cd WatchSource:0}: Error finding container d9d587fd92f25bb7dbd41df300f913a9cc49c39677c7217a4fbc014ebfb600cd: Status 404 returned error can't find the container with id d9d587fd92f25bb7dbd41df300f913a9cc49c39677c7217a4fbc014ebfb600cd
May 21 18:09:38.947432 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:38.947417    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 21 18:09:39.692601 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:39.692567    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5dg5c" event={"ID":"56f7e3f7-2529-4d95-9098-668b71ed866e","Type":"ContainerStarted","Data":"52a169674c0d56d6677143587ab26e88a2905d035056d83c86721641c34716f2"}
May 21 18:09:39.692987 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:39.692609    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5dg5c" event={"ID":"56f7e3f7-2529-4d95-9098-668b71ed866e","Type":"ContainerStarted","Data":"d9d587fd92f25bb7dbd41df300f913a9cc49c39677c7217a4fbc014ebfb600cd"}
May 21 18:09:39.692987 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:39.692686    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5dg5c"
May 21 18:09:39.714462 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:39.714077    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5dg5c" podStartSLOduration=1.714060822 podStartE2EDuration="1.714060822s" podCreationTimestamp="2026-05-21 18:09:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 18:09:39.713077672 +0000 UTC m=+1619.823470651" watchObservedRunningTime="2026-05-21 18:09:39.714060822 +0000 UTC m=+1619.824453803"
May 21 18:09:50.698160 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:09:50.698129    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-5dg5c"
May 21 18:12:40.507186 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:12:40.507079    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 18:12:40.511243 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:12:40.510113    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 18:17:40.530155 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:17:40.530046    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 18:17:40.533669 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:17:40.533248    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 18:19:28.885121 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:28.885088    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-nh5d7_f213e0dc-1d4d-46d6-a740-e1f44d9c2471/manager/0.log"
May 21 18:19:29.015384 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:29.015353    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-84c48d449b-q96vd_bc0d0284-fd95-4b5e-8986-a24e59c951e0/maas-api/0.log"
May 21 18:19:29.508041 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:29.508010    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-nh67r_e2c025d5-a928-437c-8a6a-26f8be22402b/manager/2.log"
May 21 18:19:29.758846 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:29.758757    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b676496df-mhnf4_2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6/manager/0.log"
May 21 18:19:29.995989 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:29.995966    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-zfcb6_608888ad-feb3-4a72-a4cb-651217086174/postgres/0.log"
May 21 18:19:31.973640 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:31.973613    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-5dg5c_56f7e3f7-2529-4d95-9098-668b71ed866e/manager/0.log"
May 21 18:19:32.718185 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:32.718155    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-4tjtk_61fbef2c-8573-4529-9c2f-e7e166e5104c/discovery/0.log"
May 21 18:19:32.947711 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:32.947683    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54f55555d9-tkwmc_b13d3629-00bd-42cf-b91e-b713b014618e/kube-auth-proxy/0.log"
May 21 18:19:33.072389 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:33.072315    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-mnf2l_b36bcdb0-9066-4400-83d2-0bd313fa07e4/istio-proxy/0.log"
May 21 18:19:33.688554 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:33.688529    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv_5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf/storage-initializer/0.log"
May 21 18:19:33.696327 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:33.696304    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-5clpv_5f3bc2a1-79cc-42f8-9f4c-5536bf1f59cf/main/0.log"
May 21 18:19:33.823087 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:33.823063    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4_84cc45f8-0474-479a-bf08-b8b9fe1bbe8d/storage-initializer/0.log"
May 21 18:19:33.831697 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:33.831674    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-2vkr4_84cc45f8-0474-479a-bf08-b8b9fe1bbe8d/main/0.log"
May 21 18:19:33.957413 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:33.957334    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-bcn2q_4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a/storage-initializer/0.log"
May 21 18:19:33.967406 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:33.967385    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-bcn2q_4fc851e8-0c60-4e3d-b5f2-ed2d6368d60a/main/0.log"
May 21 18:19:41.714880 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:41.714851    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5qhm9_dcb17a36-6e79-49f3-a00d-840467406c82/global-pull-secret-syncer/0.log"
May 21 18:19:41.855138 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:41.855108    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-brblc_0b01317a-6e08-4b66-838d-59b9b97f25ef/konnectivity-agent/0.log"
May 21 18:19:41.987175 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:41.987094    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-125.ec2.internal_49c69236e53b609f3a6575a47767eea0/haproxy/0.log"
May 21 18:19:46.957470 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:46.957437    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-5dg5c_56f7e3f7-2529-4d95-9098-668b71ed866e/manager/0.log"
May 21 18:19:48.763468 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:48.763429    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b2wqm_417f466c-6a1b-4811-a948-424e72817c1f/node-exporter/0.log"
May 21 18:19:48.801595 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:48.801564    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b2wqm_417f466c-6a1b-4811-a948-424e72817c1f/kube-rbac-proxy/0.log"
May 21 18:19:48.823460 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:48.823437    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b2wqm_417f466c-6a1b-4811-a948-424e72817c1f/init-textfile/0.log"
May 21 18:19:50.256806 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.256754    2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"]
May 21 18:19:50.259807 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.259774    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.264042 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.264012    2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b2w9r\"/\"default-dockercfg-s5bsx\""
May 21 18:19:50.264042 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.264034    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b2w9r\"/\"kube-root-ca.crt\""
May 21 18:19:50.269770 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.264585    2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b2w9r\"/\"openshift-service-ca.crt\""
May 21 18:19:50.269770 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.268573    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"]
May 21 18:19:50.289257 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.289221    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f349277f-1a1f-4601-a466-951605f5dc86-podres\") pod \"perf-node-gather-daemonset-jnp4t\" (UID: \"f349277f-1a1f-4601-a466-951605f5dc86\") " pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.289257 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.289261    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f349277f-1a1f-4601-a466-951605f5dc86-sys\") pod \"perf-node-gather-daemonset-jnp4t\" (UID: \"f349277f-1a1f-4601-a466-951605f5dc86\") " pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.289466 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.289330    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f349277f-1a1f-4601-a466-951605f5dc86-proc\") pod \"perf-node-gather-daemonset-jnp4t\" (UID: \"f349277f-1a1f-4601-a466-951605f5dc86\") " pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.289466 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.289377    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzzwf\" (UniqueName: \"kubernetes.io/projected/f349277f-1a1f-4601-a466-951605f5dc86-kube-api-access-gzzwf\") pod \"perf-node-gather-daemonset-jnp4t\" (UID: \"f349277f-1a1f-4601-a466-951605f5dc86\") " pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.289466 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.289424    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f349277f-1a1f-4601-a466-951605f5dc86-lib-modules\") pod \"perf-node-gather-daemonset-jnp4t\" (UID: \"f349277f-1a1f-4601-a466-951605f5dc86\") " pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.390257 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.390215    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f349277f-1a1f-4601-a466-951605f5dc86-sys\") pod \"perf-node-gather-daemonset-jnp4t\" (UID: \"f349277f-1a1f-4601-a466-951605f5dc86\") " pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.390257 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.390266    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f349277f-1a1f-4601-a466-951605f5dc86-proc\") pod \"perf-node-gather-daemonset-jnp4t\" (UID: \"f349277f-1a1f-4601-a466-951605f5dc86\") " pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.390484 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.390291    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gzzwf\" (UniqueName: \"kubernetes.io/projected/f349277f-1a1f-4601-a466-951605f5dc86-kube-api-access-gzzwf\") pod \"perf-node-gather-daemonset-jnp4t\" (UID: \"f349277f-1a1f-4601-a466-951605f5dc86\") " pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.390484 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.390333    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f349277f-1a1f-4601-a466-951605f5dc86-lib-modules\") pod \"perf-node-gather-daemonset-jnp4t\" (UID: \"f349277f-1a1f-4601-a466-951605f5dc86\") " pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.390484 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.390346    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f349277f-1a1f-4601-a466-951605f5dc86-sys\") pod \"perf-node-gather-daemonset-jnp4t\" (UID: \"f349277f-1a1f-4601-a466-951605f5dc86\") " pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.390484 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.390381    2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f349277f-1a1f-4601-a466-951605f5dc86-podres\") pod \"perf-node-gather-daemonset-jnp4t\" (UID: \"f349277f-1a1f-4601-a466-951605f5dc86\") " pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.390484 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.390424    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f349277f-1a1f-4601-a466-951605f5dc86-proc\") pod \"perf-node-gather-daemonset-jnp4t\" (UID: \"f349277f-1a1f-4601-a466-951605f5dc86\") " pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.390484 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.390468    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f349277f-1a1f-4601-a466-951605f5dc86-lib-modules\") pod \"perf-node-gather-daemonset-jnp4t\" (UID: \"f349277f-1a1f-4601-a466-951605f5dc86\") " pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.390484 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.390482    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f349277f-1a1f-4601-a466-951605f5dc86-podres\") pod \"perf-node-gather-daemonset-jnp4t\" (UID: \"f349277f-1a1f-4601-a466-951605f5dc86\") " pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.398154 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.398129    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzzwf\" (UniqueName: \"kubernetes.io/projected/f349277f-1a1f-4601-a466-951605f5dc86-kube-api-access-gzzwf\") pod \"perf-node-gather-daemonset-jnp4t\" (UID: \"f349277f-1a1f-4601-a466-951605f5dc86\") " pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.553124 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.553046    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-gwldh_e505019e-ed7b-4564-861e-17d31ff967f5/networking-console-plugin/0.log"
May 21 18:19:50.576494 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.576460    2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:50.711889 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.711856    2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"]
May 21 18:19:50.715002 ip-10-0-135-125 kubenswrapper[2558]: W0521 18:19:50.714973    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf349277f_1a1f_4601_a466_951605f5dc86.slice/crio-30070266e3fe13390dc8ee83ad61a7668f7ee3d87f13400dcc43f8a27d11478e WatchSource:0}: Error finding container 30070266e3fe13390dc8ee83ad61a7668f7ee3d87f13400dcc43f8a27d11478e: Status 404 returned error can't find the container with id 30070266e3fe13390dc8ee83ad61a7668f7ee3d87f13400dcc43f8a27d11478e
May 21 18:19:50.716978 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:50.716955    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 21 18:19:51.716717 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:51.716678    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t" event={"ID":"f349277f-1a1f-4601-a466-951605f5dc86","Type":"ContainerStarted","Data":"03974b2068fa7ab8c9d701db426d434d71ba9650cf00a0270c8bfd486344de48"}
May 21 18:19:51.717126 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:51.716722    2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t" event={"ID":"f349277f-1a1f-4601-a466-951605f5dc86","Type":"ContainerStarted","Data":"30070266e3fe13390dc8ee83ad61a7668f7ee3d87f13400dcc43f8a27d11478e"}
May 21 18:19:51.717126 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:51.716756    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:51.734861 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:51.734816    2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t" podStartSLOduration=1.73480178 podStartE2EDuration="1.73480178s" podCreationTimestamp="2026-05-21 18:19:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-21 18:19:51.732842704 +0000 UTC m=+2231.843235684" watchObservedRunningTime="2026-05-21 18:19:51.73480178 +0000 UTC m=+2231.845194754"
May 21 18:19:52.953119 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:52.953091    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-phkwf_ceaa27ad-561f-481b-aa5d-59f9c8448199/dns/0.log"
May 21 18:19:52.973253 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:52.973226    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-phkwf_ceaa27ad-561f-481b-aa5d-59f9c8448199/kube-rbac-proxy/0.log"
May 21 18:19:53.060462 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:53.060435    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-h9lc4_f95b9ee8-6015-4709-ac02-d7db0b48a238/dns-node-resolver/0.log"
May 21 18:19:53.668020 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:53.667991    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zcbrp_511e916d-ee83-4b83-a9e4-811ff6092f9a/node-ca/0.log"
May 21 18:19:54.705660 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:54.705627    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-4tjtk_61fbef2c-8573-4529-9c2f-e7e166e5104c/discovery/0.log"
May 21 18:19:54.757366 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:54.757337    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54f55555d9-tkwmc_b13d3629-00bd-42cf-b91e-b713b014618e/kube-auth-proxy/0.log"
May 21 18:19:54.811980 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:54.811940    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-mnf2l_b36bcdb0-9066-4400-83d2-0bd313fa07e4/istio-proxy/0.log"
May 21 18:19:55.437463 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:55.437435    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dd2vd_ec348fe3-0beb-4db4-be82-ec7970b5c9e9/serve-healthcheck-canary/0.log"
May 21 18:19:56.102424 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:56.102397    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mwzc6_8121325a-69bd-4adb-84ca-f10077e94863/kube-rbac-proxy/0.log"
May 21 18:19:56.123321 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:56.123297    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mwzc6_8121325a-69bd-4adb-84ca-f10077e94863/exporter/0.log"
May 21 18:19:56.145442 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:56.145403    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mwzc6_8121325a-69bd-4adb-84ca-f10077e94863/extractor/0.log"
May 21 18:19:57.729233 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:57.729207    2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b2w9r/perf-node-gather-daemonset-jnp4t"
May 21 18:19:58.098100 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:58.097973    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-nh5d7_f213e0dc-1d4d-46d6-a740-e1f44d9c2471/manager/0.log"
May 21 18:19:58.132551 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:58.132523    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-84c48d449b-q96vd_bc0d0284-fd95-4b5e-8986-a24e59c951e0/maas-api/0.log"
May 21 18:19:58.272460 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:58.272428    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-nh67r_e2c025d5-a928-437c-8a6a-26f8be22402b/manager/1.log"
May 21 18:19:58.283178 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:58.283148    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-nh67r_e2c025d5-a928-437c-8a6a-26f8be22402b/manager/2.log"
May 21 18:19:58.346499 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:58.346475    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b676496df-mhnf4_2d50c4a3-7bed-40fe-b6a8-f1f64bc907e6/manager/0.log"
May 21 18:19:58.394263 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:19:58.394194    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-zfcb6_608888ad-feb3-4a72-a4cb-651217086174/postgres/0.log"
May 21 18:20:05.737642 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:05.737613    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qtxx2_22e5cf05-8688-49c7-8390-94b6030b614c/kube-multus-additional-cni-plugins/0.log"
May 21 18:20:05.757242 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:05.757216    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qtxx2_22e5cf05-8688-49c7-8390-94b6030b614c/egress-router-binary-copy/0.log"
May 21 18:20:05.776118 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:05.776091    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qtxx2_22e5cf05-8688-49c7-8390-94b6030b614c/cni-plugins/0.log"
May 21 18:20:05.794734 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:05.794716    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qtxx2_22e5cf05-8688-49c7-8390-94b6030b614c/bond-cni-plugin/0.log"
May 21 18:20:05.815328 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:05.815305    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qtxx2_22e5cf05-8688-49c7-8390-94b6030b614c/routeoverride-cni/0.log"
May 21 18:20:05.834694 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:05.834674    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qtxx2_22e5cf05-8688-49c7-8390-94b6030b614c/whereabouts-cni-bincopy/0.log"
May 21 18:20:05.854262 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:05.854241    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qtxx2_22e5cf05-8688-49c7-8390-94b6030b614c/whereabouts-cni/0.log"
May 21 18:20:05.921325 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:05.921302    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mkh56_89c52e7d-700e-4c52-ad88-9a33c75f8f95/kube-multus/0.log"
May 21 18:20:06.007530 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:06.007457    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sk2jf_48a7b5cd-49ef-4177-83f1-80c264282bf9/network-metrics-daemon/0.log"
May 21 18:20:06.026445 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:06.026419    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sk2jf_48a7b5cd-49ef-4177-83f1-80c264282bf9/kube-rbac-proxy/0.log"
May 21 18:20:06.875593 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:06.875568    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-controller/0.log"
May 21 18:20:06.895393 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:06.895370    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/0.log"
May 21 18:20:06.904276 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:06.904256    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovn-acl-logging/1.log"
May 21 18:20:06.924692 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:06.924671    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/kube-rbac-proxy-node/0.log"
May 21 18:20:06.944439 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:06.944410    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/kube-rbac-proxy-ovn-metrics/0.log"
May 21 18:20:06.962154 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:06.962133    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/northd/0.log"
May 21 18:20:06.983074 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:06.983054    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/nbdb/0.log"
May 21 18:20:07.002674 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:07.002644    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/sbdb/0.log"
May 21 18:20:07.099728 ip-10-0-135-125 kubenswrapper[2558]: I0521 18:20:07.099701    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ht7c_4f065c68-65f3-4014-8417-31619fc4d975/ovnkube-controller/0.log"