May 28 17:17:15.643961 ip-10-0-139-178 systemd[1]: Starting Kubernetes Kubelet...
May 28 17:17:16.074181 ip-10-0-139-178 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 17:17:16.074181 ip-10-0-139-178 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 28 17:17:16.074181 ip-10-0-139-178 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 17:17:16.074181 ip-10-0-139-178 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 28 17:17:16.074181 ip-10-0-139-178 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 17:17:16.075093 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.074995    2561 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 28 17:17:16.081020 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.080996    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 17:17:16.081020 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081016    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 17:17:16.081020 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081021    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 17:17:16.081020 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081024    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 17:17:16.081020 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081028    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081031    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081033    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081036    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081041    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081045    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081048    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081051    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081054    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081057    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081059    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081062    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081064    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081067    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081069    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081072    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081074    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081077    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081079    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 17:17:16.081228 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081082    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081084    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081087    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081089    2561 feature_gate.go:328] unrecognized feature gate: Example
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081092    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081094    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081097    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081099    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081109    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081112    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081114    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081116    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081119    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081121    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081123    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081127    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081129    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081132    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081135    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081138    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 17:17:16.081671 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081142    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081147    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081150    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081153    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081155    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081158    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081160    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081163    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081166    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081169    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081172    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081175    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081178    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081180    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081183    2561 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081185    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081188    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081190    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081193    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081195    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 17:17:16.082218 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081198    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081200    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081203    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081205    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081208    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081210    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081213    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081216    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081219    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081221    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081224    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081226    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081229    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081232    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081234    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081242    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081246    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081248    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081251    2561 feature_gate.go:328] unrecognized feature gate: Example2
May 28 17:17:16.082733 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081253    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081256    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081258    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081261    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081653    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081658    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081661    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081663    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081666    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081669    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081671    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081690    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081693    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081696    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081698    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081701    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081705    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081707    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081710    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 17:17:16.083199 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081713    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081717    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081720    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081722    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081725    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081727    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081730    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081733    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081736    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081738    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081741    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081743    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081746    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081748    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081751    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081754    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081756    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081758    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081761    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081763    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 17:17:16.083657 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081766    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081768    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081771    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081774    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081776    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081779    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081781    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081784    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081786    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081789    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081791    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081794    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081797    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081799    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081802    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081804    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081807    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081809    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081812    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 17:17:16.084177 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081815    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081818    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081823    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081827    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081830    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081833    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081835    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081838    2561 feature_gate.go:328] unrecognized feature gate: Example2
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081841    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081844    2561 feature_gate.go:328] unrecognized feature gate: Example
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081847    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081849    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081852    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081854    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081856    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081859    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081862    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081864    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081867    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081869    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 17:17:16.084636 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081871    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081874    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081878    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081880    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081883    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081885    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081888    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081890    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081893    2561 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081895    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081898    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.081900    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083079    2561 flags.go:64] FLAG: --address="0.0.0.0"
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083089    2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083097    2561 flags.go:64] FLAG: --anonymous-auth="true"
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083101    2561 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083106    2561 flags.go:64] FLAG: --authentication-token-webhook="false"
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083110    2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083114    2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083119    2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083122    2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 28 17:17:16.085149 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083125    2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083129    2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083132    2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083135    2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083138    2561 flags.go:64] FLAG: --cgroup-root=""
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083141    2561 flags.go:64] FLAG: --cgroups-per-qos="true"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083143    2561 flags.go:64] FLAG: --client-ca-file=""
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083146    2561 flags.go:64] FLAG: --cloud-config=""
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083149    2561 flags.go:64] FLAG: --cloud-provider="external"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083151    2561 flags.go:64] FLAG: --cluster-dns="[]"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083155    2561 flags.go:64] FLAG: --cluster-domain=""
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083158    2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083161    2561 flags.go:64] FLAG: --config-dir=""
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083164    2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083167    2561 flags.go:64] FLAG: --container-log-max-files="5"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083171    2561 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083174    2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083177    2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083180    2561 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083183    2561 flags.go:64] FLAG: --contention-profiling="false"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083186    2561 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083189    2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083192    2561 flags.go:64] FLAG: --cpu-manager-policy="none"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083194    2561 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083199    2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 28 17:17:16.085656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083202    2561 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083205    2561 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083208    2561 flags.go:64] FLAG: --enable-load-reader="false"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083212    2561 flags.go:64] FLAG: --enable-server="true"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083215    2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083222    2561 flags.go:64] FLAG: --event-burst="100"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083225    2561 flags.go:64] FLAG: --event-qps="50"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083228    2561 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083231    2561 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083234    2561 flags.go:64] FLAG: --eviction-hard=""
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083238    2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083241    2561 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083244    2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083247    2561 flags.go:64] FLAG: --eviction-soft=""
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083249    2561 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083252    2561 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083255    2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083258    2561 flags.go:64] FLAG: --experimental-mounter-path=""
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083260    2561 flags.go:64] FLAG: --fail-cgroupv1="false"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083263    2561 flags.go:64] FLAG: --fail-swap-on="true"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083266    2561 flags.go:64] FLAG: --feature-gates=""
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083270    2561 flags.go:64] FLAG: --file-check-frequency="20s"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083273    2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083276    2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083279    2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083282    2561 flags.go:64] FLAG: --healthz-port="10248"
May 28 17:17:16.086320 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083284    2561 flags.go:64] FLAG: --help="false"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083288    2561 flags.go:64] FLAG: --hostname-override="ip-10-0-139-178.ec2.internal"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083291    2561 flags.go:64] FLAG: --housekeeping-interval="10s"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083294    2561 flags.go:64] FLAG: --http-check-frequency="20s"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083297    2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083300    2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083303    2561 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083306    2561 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083309    2561 flags.go:64] FLAG: --image-service-endpoint=""
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083311    2561 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083315    2561 flags.go:64] FLAG: --kube-api-burst="100"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083318    2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083321    2561 flags.go:64] FLAG: --kube-api-qps="50"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083324    2561 flags.go:64] FLAG: --kube-reserved=""
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083327    2561 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083330    2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083333    2561 flags.go:64] FLAG: --kubelet-cgroups=""
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083335    2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083338    2561 flags.go:64] FLAG: --lock-file=""
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083341    2561 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083343    2561 flags.go:64] FLAG: --log-flush-frequency="5s"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083346    2561 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083351    2561 flags.go:64] FLAG: --log-json-split-stream="false"
May 28 17:17:16.086970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083354    2561 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083357    2561 flags.go:64] FLAG: --log-text-split-stream="false"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083359    2561 flags.go:64] FLAG: --logging-format="text"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083362    2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083365    2561 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083368    2561 flags.go:64] FLAG: --manifest-url=""
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083371    2561 flags.go:64] FLAG: --manifest-url-header=""
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083375    2561 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083378    2561 flags.go:64] FLAG: --max-open-files="1000000"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083382    2561 flags.go:64] FLAG: --max-pods="110"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083385    2561 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083388    2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083391    2561 flags.go:64] FLAG: --memory-manager-policy="None"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083394    2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083397    2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083399    2561 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083402    2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083411    2561 flags.go:64] FLAG: --node-status-max-images="50"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083414    2561 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083416    2561 flags.go:64] FLAG: --oom-score-adj="-999"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083420    2561 flags.go:64] FLAG: --pod-cidr=""
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083423    2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083429    2561 flags.go:64] FLAG: --pod-manifest-path=""
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083432    2561 flags.go:64] FLAG: --pod-max-pids="-1"
May 28 17:17:16.087593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083435    2561 flags.go:64] FLAG: --pods-per-core="0"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083438    2561 flags.go:64] FLAG: --port="10250"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083441    2561 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083443    2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-051c3c09bdc800e9a"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083447    2561 flags.go:64] FLAG: --qos-reserved=""
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083449    2561 flags.go:64] FLAG: --read-only-port="10255"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083452    2561 flags.go:64] FLAG: --register-node="true"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083455    2561 flags.go:64] FLAG: --register-schedulable="true"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083458    2561 flags.go:64] FLAG: --register-with-taints=""
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083461    2561 flags.go:64] FLAG: --registry-burst="10"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083468    2561 flags.go:64] FLAG: --registry-qps="5"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083470    2561 flags.go:64] FLAG: --reserved-cpus=""
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083473    2561 flags.go:64] FLAG: --reserved-memory=""
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083476    2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083479    2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083482    2561 flags.go:64] FLAG: --rotate-certificates="false"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083485    2561 flags.go:64] FLAG: --rotate-server-certificates="false"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083487    2561 flags.go:64] FLAG: --runonce="false"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083490    2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083493    2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083496    2561 flags.go:64] FLAG: --seccomp-default="false"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083498    2561 flags.go:64] FLAG: --serialize-image-pulls="true"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083501    2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083504    2561 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083507    2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083510    2561 flags.go:64] FLAG: --storage-driver-password="root"
May 28 17:17:16.088199 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083513    2561 flags.go:64] FLAG: --storage-driver-secure="false"
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083515    2561 flags.go:64] FLAG: --storage-driver-table="stats"
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083518    2561 flags.go:64] FLAG: --storage-driver-user="root"
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083522    2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083525    2561 flags.go:64] FLAG: --sync-frequency="1m0s"
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083528    2561 flags.go:64] FLAG: --system-cgroups=""
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083530    2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083535    2561 flags.go:64] FLAG: --system-reserved-cgroup=""
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083538    2561 flags.go:64] FLAG: --tls-cert-file=""
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083541    2561 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083545    2561 flags.go:64] FLAG: --tls-min-version=""
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083547    2561 flags.go:64] FLAG: --tls-private-key-file=""
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083550    2561 flags.go:64] FLAG: --topology-manager-policy="none"
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083553    2561 flags.go:64] FLAG: --topology-manager-policy-options=""
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083555    2561 flags.go:64] FLAG: --topology-manager-scope="container"
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083558    2561 flags.go:64] FLAG: --v="2"
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083563    2561 flags.go:64] FLAG: --version="false"
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083567    2561 flags.go:64] FLAG: --vmodule=""
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083571    2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.083574    2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083667    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083671    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083690    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083694    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 17:17:16.088833 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083698    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083703    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083705    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083708    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083710    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083713    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083715    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083718    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083720    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083723    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083725    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083728    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083735    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083738    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083741    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083744    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083747    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083750    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083752    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 17:17:16.089448 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083756    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083758    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083761    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083764    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083767    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083770    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083773    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083775    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083778    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083780    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083783    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083785    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083788    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083790    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083793    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083795    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083798    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083800    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083802    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083805    2561 feature_gate.go:328] unrecognized feature gate: Example2
May 28 17:17:16.089999 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083808    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083810    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083813    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083815    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083818    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083822    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083825    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083827    2561 feature_gate.go:328] unrecognized feature gate: Example
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083830    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083832    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083835    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083837    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083841    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083845    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083847    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083850    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083852    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083856    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083860    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083862    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 17:17:16.090503 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083865    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083867    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083869    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083872    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083874    2561 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083877    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083879    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083881    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083884    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083886    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083888    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083891    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083893    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083896    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083898    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083900    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083903    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083907    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083909    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083912    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 17:17:16.091007 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083915    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 17:17:16.091497 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083917    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 17:17:16.091497 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.083919    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 17:17:16.091497 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.084965    2561 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 17:17:16.091728 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.091591    2561 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 28 17:17:16.091766 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.091730    2561 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 28 17:17:16.091794 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091780    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 17:17:16.091794 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091786    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 17:17:16.091794 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091790    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 17:17:16.091794 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091795    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091798    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091801    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091804    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091806    2561 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091809    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091811    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091814    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091817    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091819    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091821    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091824    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091827    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091829    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091831    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091834    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091836    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091839    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091841    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091843    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 17:17:16.091898 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091846    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091848    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091850    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091853    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091855    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091858    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091860    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091863    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091866    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091869    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091871    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091874    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091877    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091879    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091882    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091885    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091887    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091890    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091893    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091895    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 17:17:16.092399 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091898    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091900    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091903    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091906    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091908    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091911    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091914    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091917    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091920    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091922    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091925    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091927    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091929    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091932    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091934    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091936    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091940    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091944    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091947    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 17:17:16.092918 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091949    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091952    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091955    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091957    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091960    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091962    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091964    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091968    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091970    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091973    2561 feature_gate.go:328] unrecognized feature gate: Example2
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091976    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091978    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091981    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091983    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091986    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091988    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091991    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091993    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091995    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.091998    2561 feature_gate.go:328] unrecognized feature gate: Example
May 28 17:17:16.093384 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092000    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 17:17:16.093902 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092003    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 17:17:16.093902 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092005    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 17:17:16.093902 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092007    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 17:17:16.093902 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.092012    2561 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 17:17:16.093902 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092133    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 17:17:16.093902 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092139    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 17:17:16.093902 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092142    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 17:17:16.093902 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092144    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 17:17:16.093902 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092147    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 17:17:16.093902 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092151    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 17:17:16.093902 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092155    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 17:17:16.093902 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092157    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 17:17:16.093902 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092160    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 17:17:16.093902 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092163    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 17:17:16.093902 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092166    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092168    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092172    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092175    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092179    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092182    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092185    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092188    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092190    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092193    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092195    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092198    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092201    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092203    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092205    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092208    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092210    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092213    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092215    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 17:17:16.094281 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092218    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092221    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092223    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092225    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092228    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092230    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092233    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092235    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092238    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092240    2561 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092242    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092245    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092247    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092249    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092252    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092255    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092258    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092261    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092264    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092266    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 17:17:16.094777 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092269    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092271    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092273    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092276    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092293    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092297    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092300    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092303    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092306    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092309    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092312    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092314    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092317    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092320    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092322    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092325    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092327    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092330    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092332    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092334    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 17:17:16.095256 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092337    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 17:17:16.095775 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092339    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 17:17:16.095775 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092341    2561 feature_gate.go:328] unrecognized feature gate: Example
May 28 17:17:16.095775 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092344    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 17:17:16.095775 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092346    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 17:17:16.095775 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092348    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 17:17:16.095775 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092351    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 17:17:16.095775 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092355    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 17:17:16.095775 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092357    2561 feature_gate.go:328] unrecognized feature gate: Example2
May 28 17:17:16.095775 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092360    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 17:17:16.095775 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092363    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 17:17:16.095775 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092366    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 17:17:16.095775 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092368    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 17:17:16.095775 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092371    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 17:17:16.095775 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092373    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 17:17:16.095775 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092376    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 17:17:16.095775 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:16.092378    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 17:17:16.096180 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.092382    2561 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 17:17:16.096180 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.093022    2561 server.go:962] "Client rotation is on, will bootstrap in background"
May 28 17:17:16.096180 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.095573    2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 28 17:17:16.096543 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.096531    2561 server.go:1019] "Starting client certificate rotation"
May 28 17:17:16.097022 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.097003    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 17:17:16.097065 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.097042    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 17:17:16.119311 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.119281    2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 17:17:16.121140 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.121110    2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 17:17:16.139107 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.139084    2561 log.go:25] "Validated CRI v1 runtime API"
May 28 17:17:16.143465 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.143446    2561 log.go:25] "Validated CRI v1 image API"
May 28 17:17:16.144553 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.144539    2561 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 28 17:17:16.147286 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.147250    2561 fs.go:135] Filesystem UUIDs: map[262f8f8d-c6e4-4ad9-b5de-83776b89320c:/dev/nvme0n1p4 2e9d5761-dc59-41a8-a11b-b5bdfc2e2bcf:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
May 28 17:17:16.147286 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.147275    2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 28 17:17:16.151947 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.151821    2561 manager.go:217] Machine: {Timestamp:2026-05-28 17:17:16.151316195 +0000 UTC m=+0.390866576 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3110804 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec293945f5654c2121fa275e9e9f1688 SystemUUID:ec293945-f565-4c21-21fa-275e9e9f1688 BootID:4099e941-259c-4e67-9b42-cd1c271e330e Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2a:79:59:9a:57 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2a:79:59:9a:57 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:d3:11:e9:84:69 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 28 17:17:16.153789 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.153775    2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 28 17:17:16.153943 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.153925    2561 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 28 17:17:16.154015 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.153942    2561 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 17:17:16.155813 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.155785    2561 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 28 17:17:16.155995 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.155814    2561 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-178.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 28 17:17:16.156077 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.156011    2561 topology_manager.go:138] "Creating topology manager with none policy"
May 28 17:17:16.156077 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.156026    2561 container_manager_linux.go:306] "Creating device plugin manager"
May 28 17:17:16.156077 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.156044    2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 17:17:16.156660 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.156648    2561 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 17:17:16.158572 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.158559    2561 state_mem.go:36] "Initialized new in-memory state store"
May 28 17:17:16.158721 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.158711    2561 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 28 17:17:16.161288 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.161276    2561 kubelet.go:491] "Attempting to sync node with API server"
May 28 17:17:16.161351 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.161300    2561 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 28 17:17:16.161351 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.161318    2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 28 17:17:16.161351 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.161330    2561 kubelet.go:397] "Adding apiserver pod source"
May 28 17:17:16.161351 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.161343    2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 28 17:17:16.162560 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.162547    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 17:17:16.162622 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.162570    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 17:17:16.169044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.165333    2561 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 28 17:17:16.169044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.167007    2561 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 28 17:17:16.169044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.168189    2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 28 17:17:16.169044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.168205    2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 28 17:17:16.169044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.168211    2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 28 17:17:16.169044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.168216    2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 28 17:17:16.169044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.168224    2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 28 17:17:16.169044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.168230    2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 28 17:17:16.169044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.168239    2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 28 17:17:16.169044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.168248    2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 28 17:17:16.169044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.168255    2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 28 17:17:16.169044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.168261    2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 28 17:17:16.169044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.168274    2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 28 17:17:16.169044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.168280    2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 28 17:17:16.169044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.168921    2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 28 17:17:16.169044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.168928    2561 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 28 17:17:16.175451 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.175436    2561 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 28 17:17:16.175510 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.175468    2561 server.go:1295] "Started kubelet"
May 28 17:17:16.175884 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.175867    2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 28 17:17:16.176156 ip-10-0-139-178 systemd[1]: Started Kubernetes Kubelet.
May 28 17:17:16.177290 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.177187    2561 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 28 17:17:16.177364 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.177321    2561 server_v1.go:47] "podresources" method="list" useActivePods=true
May 28 17:17:16.177364 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.177325    2561 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 28 17:17:16.177469 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.177428    2561 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 28 17:17:16.177605 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.177275    2561 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-178.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 28 17:17:16.179573 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.179552    2561 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 28 17:17:16.180863 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.180833    2561 server.go:317] "Adding debug handlers to kubelet server"
May 28 17:17:16.181914 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.181885    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pwrp2"
May 28 17:17:16.184474 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.184455    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 28 17:17:16.185361 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.184954    2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 28 17:17:16.185361 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.185111    2561 volume_manager.go:295] "The desired_state_of_world populator starts"
May 28 17:17:16.185361 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.185126    2561 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 28 17:17:16.185361 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.185234    2561 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 28 17:17:16.185361 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.185314    2561 reconstruct.go:97] "Volume reconstruction finished"
May 28 17:17:16.185361 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.185326    2561 reconciler.go:26] "Reconciler: start to sync state"
May 28 17:17:16.185696 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.185663    2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found"
May 28 17:17:16.186218 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.185131    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-178.ec2.internal.18b3ca3bdb783aa6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-178.ec2.internal,UID:ip-10-0-139-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-178.ec2.internal,},FirstTimestamp:2026-05-28 17:17:16.175448742 +0000 UTC m=+0.414999123,LastTimestamp:2026-05-28 17:17:16.175448742 +0000 UTC m=+0.414999123,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-178.ec2.internal,}"
May 28 17:17:16.186513 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.186494    2561 factory.go:153] Registering CRI-O factory
May 28 17:17:16.186587 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.186546    2561 factory.go:223] Registration of the crio container factory successfully
May 28 17:17:16.186648 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.186601    2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 28 17:17:16.186648 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.186608    2561 factory.go:55] Registering systemd factory
May 28 17:17:16.186648 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.186614    2561 factory.go:223] Registration of the systemd container factory successfully
May 28 17:17:16.186648 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.186628    2561 factory.go:103] Registering Raw factory
May 28 17:17:16.186648 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.186638    2561 manager.go:1196] Started watching for new ooms in manager
May 28 17:17:16.186962 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.186950    2561 manager.go:319] Starting recovery of all containers
May 28 17:17:16.187338 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.187314    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pwrp2"
May 28 17:17:16.187733 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.187716    2561 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 28 17:17:16.195501 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.195481    2561 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 17:17:16.196325 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.196307    2561 manager.go:324] Recovery completed
May 28 17:17:16.197490 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.197473    2561 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-178.ec2.internal\" not found" node="ip-10-0-139-178.ec2.internal"
May 28 17:17:16.197846 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.197827    2561 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
May 28 17:17:16.200646 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.200634    2561 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:17:16.203777 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.203762    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:17:16.203836 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.203792    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:17:16.203836 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.203802    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientPID"
May 28 17:17:16.205049 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.205030    2561 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 28 17:17:16.205049 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.205047    2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 28 17:17:16.205180 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.205069    2561 state_mem.go:36] "Initialized new in-memory state store"
May 28 17:17:16.207569 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.207556    2561 policy_none.go:49] "None policy: Start"
May 28 17:17:16.207635 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.207574    2561 memory_manager.go:186] "Starting memorymanager" policy="None"
May 28 17:17:16.207635 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.207586    2561 state_mem.go:35] "Initializing new in-memory state store"
May 28 17:17:16.256449 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.247840    2561 manager.go:341] "Starting Device Plugin manager"
May 28 17:17:16.256449 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.247910    2561 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 28 17:17:16.256449 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.247924    2561 server.go:85] "Starting device plugin registration server"
May 28 17:17:16.256449 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.248174    2561 eviction_manager.go:189] "Eviction manager: starting control loop"
May 28 17:17:16.256449 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.248185    2561 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 28 17:17:16.256449 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.248278    2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 28 17:17:16.256449 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.248361    2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 28 17:17:16.256449 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.248369    2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 28 17:17:16.256449 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.249172    2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 28 17:17:16.256449 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.249204    2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-178.ec2.internal\" not found"
May 28 17:17:16.314988 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.314950    2561 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 28 17:17:16.316422 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.316401    2561 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 28 17:17:16.316553 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.316433    2561 status_manager.go:230] "Starting to sync pod status with apiserver"
May 28 17:17:16.316553 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.316452    2561 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 28 17:17:16.316553 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.316460    2561 kubelet.go:2451] "Starting kubelet main sync loop"
May 28 17:17:16.316553 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.316550    2561 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 28 17:17:16.318865 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.318840    2561 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 17:17:16.348854 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.348807    2561 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:17:16.349761 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.349745    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:17:16.349839 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.349777    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:17:16.349839 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.349795    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientPID"
May 28 17:17:16.349839 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.349822    2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-178.ec2.internal"
May 28 17:17:16.356866 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.356844    2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-178.ec2.internal"
May 28 17:17:16.356937 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.356871    2561 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-178.ec2.internal\": node \"ip-10-0-139-178.ec2.internal\" not found"
May 28 17:17:16.374929 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.374902    2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found"
May 28 17:17:16.417297 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.417265    2561 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal"]
May 28 17:17:16.417415 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.417347    2561 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:17:16.418883 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.418860    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:17:16.418997 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.418891    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:17:16.418997 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.418903    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientPID"
May 28 17:17:16.420240 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.420227    2561 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:17:16.420394 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.420379    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal"
May 28 17:17:16.420436 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.420410    2561 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:17:16.420985 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.420966    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:17:16.421089 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.420998    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:17:16.421089 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.421014    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientPID"
May 28 17:17:16.421089 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.421021    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:17:16.421089 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.421048    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:17:16.421089 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.421062    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientPID"
May 28 17:17:16.422147 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.422134    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal"
May 28 17:17:16.422208 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.422159    2561 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:17:16.422837 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.422822    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:17:16.422919 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.422853    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:17:16.422919 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.422868    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientPID"
May 28 17:17:16.448644 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.448615    2561 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-178.ec2.internal\" not found" node="ip-10-0-139-178.ec2.internal"
May 28 17:17:16.452152 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.452134    2561 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-178.ec2.internal\" not found" node="ip-10-0-139-178.ec2.internal"
May 28 17:17:16.475947 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.475916    2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found"
May 28 17:17:16.576524 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.576494    2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found"
May 28 17:17:16.586871 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.586844    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/312feb2e8872e44b4817c5bdff3cc0eb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal\" (UID: \"312feb2e8872e44b4817c5bdff3cc0eb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal"
May 28 17:17:16.586960 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.586877    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8b6f5dedac2bb56dda3493221133ac3c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-178.ec2.internal\" (UID: \"8b6f5dedac2bb56dda3493221133ac3c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal"
May 28 17:17:16.586960 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.586895    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/312feb2e8872e44b4817c5bdff3cc0eb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal\" (UID: \"312feb2e8872e44b4817c5bdff3cc0eb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal"
May 28 17:17:16.677267 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.677232    2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found"
May 28 17:17:16.687541 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.687520    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/312feb2e8872e44b4817c5bdff3cc0eb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal\" (UID: \"312feb2e8872e44b4817c5bdff3cc0eb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal"
May 28 17:17:16.687612 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.687549    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8b6f5dedac2bb56dda3493221133ac3c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-178.ec2.internal\" (UID: \"8b6f5dedac2bb56dda3493221133ac3c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal"
May 28 17:17:16.687612 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.687565    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/312feb2e8872e44b4817c5bdff3cc0eb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal\" (UID: \"312feb2e8872e44b4817c5bdff3cc0eb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal"
May 28 17:17:16.687612 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.687605    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/312feb2e8872e44b4817c5bdff3cc0eb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal\" (UID: \"312feb2e8872e44b4817c5bdff3cc0eb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal"
May 28 17:17:16.687749 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.687635    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/312feb2e8872e44b4817c5bdff3cc0eb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal\" (UID: \"312feb2e8872e44b4817c5bdff3cc0eb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal"
May 28 17:17:16.687749 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.687660    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8b6f5dedac2bb56dda3493221133ac3c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-178.ec2.internal\" (UID: \"8b6f5dedac2bb56dda3493221133ac3c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal"
May 28 17:17:16.751703 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.751649    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal"
May 28 17:17:16.754451 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:16.754436    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal"
May 28 17:17:16.778187 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.778164    2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found"
May 28 17:17:16.878773 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.878659    2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found"
May 28 17:17:16.979182 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:16.979153    2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found"
May 28 17:17:17.079664 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:17.079612    2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found"
May 28 17:17:17.096024 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:17.096002    2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 28 17:17:17.096153 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:17.096138    2561 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 17:17:17.096193 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:17.096155    2561 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 17:17:17.180482 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:17.180412    2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found"
May 28 17:17:17.185500 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:17.185484    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 28 17:17:17.189512 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:17.189484    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 17:12:16 +0000 UTC" deadline="2028-02-19 04:09:46.96855375 +0000 UTC"
May 28 17:17:17.189512 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:17.189508    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15154h52m29.779048311s"
May 28 17:17:17.200467 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:17.200445    2561 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 17:17:17.218365 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:17.218338    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tl4w4"
May 28 17:17:17.226307 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:17.226291    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tl4w4"
May 28 17:17:17.281029 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:17.281000    2561 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found"
May 28 17:17:17.331298 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:17.331280    2561 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 17:17:17.385738 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:17.385708    2561 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal"
May 28 17:17:17.394434 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:17.394413    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 17:17:17.395786 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:17.395774    2561 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal"
May 28 17:17:17.407469 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:17.407450    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 17:17:17.429649 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:17.429423    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b6f5dedac2bb56dda3493221133ac3c.slice/crio-0c3a0d028654ee504f76e58ad7228a68946fc006a1e080fcc1dc1aa98fe12826 WatchSource:0}: Error finding container 0c3a0d028654ee504f76e58ad7228a68946fc006a1e080fcc1dc1aa98fe12826: Status 404 returned error can't find the container with id 0c3a0d028654ee504f76e58ad7228a68946fc006a1e080fcc1dc1aa98fe12826
May 28 17:17:17.429889 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:17.429869    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod312feb2e8872e44b4817c5bdff3cc0eb.slice/crio-cf6a4a972b6c6c44bcabfde5e7f1b9965062f55fb5c9037631433331b4441d2f WatchSource:0}: Error finding container cf6a4a972b6c6c44bcabfde5e7f1b9965062f55fb5c9037631433331b4441d2f: Status 404 returned error can't find the container with id cf6a4a972b6c6c44bcabfde5e7f1b9965062f55fb5c9037631433331b4441d2f
May 28 17:17:17.435158 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:17.435144    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 17:17:17.786669 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:17.786571    2561 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 17:17:18.163176 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.163132    2561 apiserver.go:52] "Watching apiserver"
May 28 17:17:18.171741 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.171708    2561 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 28 17:17:18.173491 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.173457    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal","openshift-multus/network-metrics-daemon-7xrml","openshift-network-diagnostics/network-check-target-hrcwh","openshift-ovn-kubernetes/ovnkube-node-5gxv8","kube-system/konnectivity-agent-cvsnb","kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs","openshift-dns/node-resolver-vc2ht","openshift-image-registry/node-ca-kmg5m","openshift-multus/multus-additional-cni-plugins-l55xd","openshift-multus/multus-h7tx9","openshift-network-operator/iptables-alerter-jpw2j","openshift-cluster-node-tuning-operator/tuned-cj8n6"]
May 28 17:17:18.176649 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.176223    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.176649 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.176527    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:18.176649 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:18.176628    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:18.179815 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.179635    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 28 17:17:18.179815 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.179695    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-27pb5\""
May 28 17:17:18.179815 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.179635    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 28 17:17:18.179815 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.179727    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 28 17:17:18.180129 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.180112    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:18.180193 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:18.180172    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:18.180275 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.180259    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.181592 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.181570    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cvsnb"
May 28 17:17:18.183238 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.183220    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 28 17:17:18.183408 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.183394    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 28 17:17:18.183529 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.183516    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 28 17:17:18.183831 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.183805    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 28 17:17:18.183970 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.183955    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6lq8v\""
May 28 17:17:18.184123 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.184109    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 28 17:17:18.184472 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.184454    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 28 17:17:18.184723 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.184710    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 28 17:17:18.184830 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.184810    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vc2ht"
May 28 17:17:18.184884 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.184832    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 28 17:17:18.184937 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.184926    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-p7vz4\""
May 28 17:17:18.186729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.186667    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kmg5m"
May 28 17:17:18.186807 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.186789    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.188205 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.188187    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 28 17:17:18.188298 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.188233    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sswzk\""
May 28 17:17:18.188298 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.188264    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 28 17:17:18.188939 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.188922    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.189090 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.189072    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 28 17:17:18.189665 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.189642    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 28 17:17:18.190695 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.190331    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 28 17:17:18.190695 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.190369    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jvlzp\""
May 28 17:17:18.190695 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.190503    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bx6ph\""
May 28 17:17:18.190695 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.190570    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 28 17:17:18.190695 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.190598    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 28 17:17:18.190695 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.190616    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 28 17:17:18.190695 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.190663    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 28 17:17:18.190992 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.190603    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 28 17:17:18.190992 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.190932    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jpw2j"
May 28 17:17:18.191592 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.191573    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lnx7k\""
May 28 17:17:18.191669 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.191597    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 28 17:17:18.192765 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.192740    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.193629 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.193612    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 28 17:17:18.193857 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.193842    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 28 17:17:18.193975 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.193867    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wgwlr\""
May 28 17:17:18.193975 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.193969    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 28 17:17:18.195670 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.195641    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ss4xm\""
May 28 17:17:18.195880 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.195829    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-run-netns\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.195880 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.195867    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b983f07-6127-487d-a077-b09e0d2c93f7-ovn-node-metrics-cert\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.196012 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.195892    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4222bafb-9e5f-4884-832d-83364e16f3e9-tmp-dir\") pod \"node-resolver-vc2ht\" (UID: \"4222bafb-9e5f-4884-832d-83364e16f3e9\") " pod="openshift-dns/node-resolver-vc2ht"
May 28 17:17:18.196012 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.195918    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-os-release\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.196012 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.195942    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-cni-binary-copy\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.196012 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.195964    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.196012 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.195991    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcdh4\" (UniqueName: \"kubernetes.io/projected/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-kube-api-access-xcdh4\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.196257 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196014    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwkdt\" (UniqueName: \"kubernetes.io/projected/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-kube-api-access-cwkdt\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.196257 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196040    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-kubelet\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.196257 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196063    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4222bafb-9e5f-4884-832d-83364e16f3e9-hosts-file\") pod \"node-resolver-vc2ht\" (UID: \"4222bafb-9e5f-4884-832d-83364e16f3e9\") " pod="openshift-dns/node-resolver-vc2ht"
May 28 17:17:18.196257 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196085    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cf00e197-64d4-4961-9730-248b6e1ec010-serviceca\") pod \"node-ca-kmg5m\" (UID: \"cf00e197-64d4-4961-9730-248b6e1ec010\") " pod="openshift-image-registry/node-ca-kmg5m"
May 28 17:17:18.196257 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196109    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.196257 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196135    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-socket-dir\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.196257 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196157    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-registration-dir\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.196257 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196179    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-systemd-units\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.196257 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196201    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-node-log\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.196257 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196225    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/737fe4ef-9e14-4558-8696-2598182022ab-konnectivity-ca\") pod \"konnectivity-agent-cvsnb\" (UID: \"737fe4ef-9e14-4558-8696-2598182022ab\") " pod="kube-system/konnectivity-agent-cvsnb"
May 28 17:17:18.196257 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196251    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njwl8\" (UniqueName: \"kubernetes.io/projected/4222bafb-9e5f-4884-832d-83364e16f3e9-kube-api-access-njwl8\") pod \"node-resolver-vc2ht\" (UID: \"4222bafb-9e5f-4884-832d-83364e16f3e9\") " pod="openshift-dns/node-resolver-vc2ht"
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196288    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-cnibin\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196378    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxmpn\" (UniqueName: \"kubernetes.io/projected/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-kube-api-access-vxmpn\") pod \"network-metrics-daemon-7xrml\" (UID: \"1a8e2081-5b89-4115-82b2-d1afbf6f58ed\") " pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196404    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-var-lib-openvswitch\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196445    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196475    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-run-openvswitch\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196522    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-run-ovn\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196548    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-log-socket\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196584    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-cni-bin\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196606    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-cni-netd\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.195894    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196647    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196715    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-device-dir\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196740    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-run-systemd\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196770    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8b983f07-6127-487d-a077-b09e0d2c93f7-ovnkube-script-lib\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196794    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/737fe4ef-9e14-4558-8696-2598182022ab-agent-certs\") pod \"konnectivity-agent-cvsnb\" (UID: \"737fe4ef-9e14-4558-8696-2598182022ab\") " pod="kube-system/konnectivity-agent-cvsnb"
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196822    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxnhb\" (UniqueName: \"kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb\") pod \"network-check-target-hrcwh\" (UID: \"b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4\") " pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:18.196841 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196845    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-sys-fs\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.197596 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196880    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-etc-openvswitch\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.197596 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196910    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs\") pod \"network-metrics-daemon-7xrml\" (UID: \"1a8e2081-5b89-4115-82b2-d1afbf6f58ed\") " pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:18.197596 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196938    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8b983f07-6127-487d-a077-b09e0d2c93f7-ovnkube-config\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.197596 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196961    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf00e197-64d4-4961-9730-248b6e1ec010-host\") pod \"node-ca-kmg5m\" (UID: \"cf00e197-64d4-4961-9730-248b6e1ec010\") " pod="openshift-image-registry/node-ca-kmg5m"
May 28 17:17:18.197596 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.196985    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-system-cni-dir\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.197596 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.197010    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.197596 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.197036    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-etc-selinux\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.197596 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.197061    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-slash\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.197596 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.197089    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-run-ovn-kubernetes\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.197596 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.197112    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8b983f07-6127-487d-a077-b09e0d2c93f7-env-overrides\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.197596 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.197137    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvqk2\" (UniqueName: \"kubernetes.io/projected/8b983f07-6127-487d-a077-b09e0d2c93f7-kube-api-access-bvqk2\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.197596 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.197159    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpwcz\" (UniqueName: \"kubernetes.io/projected/cf00e197-64d4-4961-9730-248b6e1ec010-kube-api-access-rpwcz\") pod \"node-ca-kmg5m\" (UID: \"cf00e197-64d4-4961-9730-248b6e1ec010\") " pod="openshift-image-registry/node-ca-kmg5m"
May 28 17:17:18.197596 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.197195    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.228329 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.228301    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 17:12:17 +0000 UTC" deadline="2027-11-22 14:12:30.611045551 +0000 UTC"
May 28 17:17:18.228442 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.228330    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13028h55m12.38271914s"
May 28 17:17:18.287083 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.287044    2561 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 28 17:17:18.297409 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297376    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-run-netns\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.297409 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297418    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4222bafb-9e5f-4884-832d-83364e16f3e9-tmp-dir\") pod \"node-resolver-vc2ht\" (UID: \"4222bafb-9e5f-4884-832d-83364e16f3e9\") " pod="openshift-dns/node-resolver-vc2ht"
May 28 17:17:18.297743 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297447    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.297743 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297482    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88mpc\" (UniqueName: \"kubernetes.io/projected/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-kube-api-access-88mpc\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.297743 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297501    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-run-netns\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.297743 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297506    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-modprobe-d\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.297743 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297564    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.297743 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297597    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-system-cni-dir\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.297743 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297623    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-multus-daemon-config\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.297743 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297662    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-run-multus-certs\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.297743 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297705    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b0b6f57-f7d5-494f-a47b-132adfbe518f-host-slash\") pod \"iptables-alerter-jpw2j\" (UID: \"8b0b6f57-f7d5-494f-a47b-132adfbe518f\") " pod="openshift-network-operator/iptables-alerter-jpw2j"
May 28 17:17:18.297743 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297730    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-sysconfig\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297751    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-lib-modules\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297778    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-socket-dir\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297805    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-systemd-units\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297831    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/737fe4ef-9e14-4558-8696-2598182022ab-konnectivity-ca\") pod \"konnectivity-agent-cvsnb\" (UID: \"737fe4ef-9e14-4558-8696-2598182022ab\") " pod="kube-system/konnectivity-agent-cvsnb"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297856    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-njwl8\" (UniqueName: \"kubernetes.io/projected/4222bafb-9e5f-4884-832d-83364e16f3e9-kube-api-access-njwl8\") pod \"node-resolver-vc2ht\" (UID: \"4222bafb-9e5f-4884-832d-83364e16f3e9\") " pod="openshift-dns/node-resolver-vc2ht"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297865    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4222bafb-9e5f-4884-832d-83364e16f3e9-tmp-dir\") pod \"node-resolver-vc2ht\" (UID: \"4222bafb-9e5f-4884-832d-83364e16f3e9\") " pod="openshift-dns/node-resolver-vc2ht"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297882    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-cnibin\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297909    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-os-release\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297968    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-var-lib-cni-bin\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.297998    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vxmpn\" (UniqueName: \"kubernetes.io/projected/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-kube-api-access-vxmpn\") pod \"network-metrics-daemon-7xrml\" (UID: \"1a8e2081-5b89-4115-82b2-d1afbf6f58ed\") " pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298025    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-var-lib-openvswitch\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298052    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-run-openvswitch\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298083    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-log-socket\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298110    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298121    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298136    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-var-lib-kubelet\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.298209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298160    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-tuned\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298195    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8b983f07-6127-487d-a077-b09e0d2c93f7-ovnkube-script-lib\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298236    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298251    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-cnibin\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298345    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-log-socket\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298378    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-var-lib-openvswitch\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298402    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-run-openvswitch\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298442    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298533    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-socket-dir\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298569    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-sys\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298602    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-etc-openvswitch\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298635    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8b983f07-6127-487d-a077-b09e0d2c93f7-ovnkube-script-lib\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298650    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-multus-socket-dir-parent\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298703    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-run-netns\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298732    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-systemd\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298788    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs\") pod \"network-metrics-daemon-7xrml\" (UID: \"1a8e2081-5b89-4115-82b2-d1afbf6f58ed\") " pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298869    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8b983f07-6127-487d-a077-b09e0d2c93f7-ovnkube-config\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.298978 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298931    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-system-cni-dir\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.299848 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:18.298956    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:17:18.299848 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298961    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-cnibin\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.299848 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298968    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/737fe4ef-9e14-4558-8696-2598182022ab-konnectivity-ca\") pod \"konnectivity-agent-cvsnb\" (UID: \"737fe4ef-9e14-4558-8696-2598182022ab\") " pod="kube-system/konnectivity-agent-cvsnb"
May 28 17:17:18.299848 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.298987    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-sysctl-d\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.299848 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:18.299002    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs podName:1a8e2081-5b89-4115-82b2-d1afbf6f58ed nodeName:}" failed. No retries permitted until 2026-05-28 17:17:18.798987537 +0000 UTC m=+3.038537905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs") pod "network-metrics-daemon-7xrml" (UID: "1a8e2081-5b89-4115-82b2-d1afbf6f58ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:17:18.299848 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299173    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.299848 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299194    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-etc-selinux\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.299848 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299242    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bvqk2\" (UniqueName: \"kubernetes.io/projected/8b983f07-6127-487d-a077-b09e0d2c93f7-kube-api-access-bvqk2\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.299848 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299267    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rpwcz\" (UniqueName: \"kubernetes.io/projected/cf00e197-64d4-4961-9730-248b6e1ec010-kube-api-access-rpwcz\") pod \"node-ca-kmg5m\" (UID: \"cf00e197-64d4-4961-9730-248b6e1ec010\") " pod="openshift-image-registry/node-ca-kmg5m"
May 28 17:17:18.299848 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299286    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-sysctl-conf\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.299848 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299301    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-run\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.299848 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299316    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snshj\" (UniqueName: \"kubernetes.io/projected/00d2dbe6-3f97-4fab-8af5-581a8329be5c-kube-api-access-snshj\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.299848 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299334    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b983f07-6127-487d-a077-b09e0d2c93f7-ovn-node-metrics-cert\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.299848 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299378    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-os-release\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.299848 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299394    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-cni-binary-copy\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.299848 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299414    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xcdh4\" (UniqueName: \"kubernetes.io/projected/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-kube-api-access-xcdh4\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299439    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-hostroot\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299458    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cwkdt\" (UniqueName: \"kubernetes.io/projected/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-kube-api-access-cwkdt\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299474    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-kubelet\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299488    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4222bafb-9e5f-4884-832d-83364e16f3e9-hosts-file\") pod \"node-resolver-vc2ht\" (UID: \"4222bafb-9e5f-4884-832d-83364e16f3e9\") " pod="openshift-dns/node-resolver-vc2ht"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299504    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cf00e197-64d4-4961-9730-248b6e1ec010-serviceca\") pod \"node-ca-kmg5m\" (UID: \"cf00e197-64d4-4961-9730-248b6e1ec010\") " pod="openshift-image-registry/node-ca-kmg5m"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299510    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8b983f07-6127-487d-a077-b09e0d2c93f7-ovnkube-config\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299520    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-run-k8s-cni-cncf-io\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299536    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-var-lib-cni-multus\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299551    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d2dbe6-3f97-4fab-8af5-581a8329be5c-tmp\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299561    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-etc-openvswitch\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299567    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-registration-dir\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299646    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-node-log\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299664    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-etc-kubernetes\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299696    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-var-lib-kubelet\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299715    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-run-ovn\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299741    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-cni-bin\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.300729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299758    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-cni-netd\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299773    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-multus-cni-dir\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299797    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-cni-binary-copy\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299813    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-kubernetes\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299831    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-device-dir\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299848    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-run-systemd\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299864    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/737fe4ef-9e14-4558-8696-2598182022ab-agent-certs\") pod \"konnectivity-agent-cvsnb\" (UID: \"737fe4ef-9e14-4558-8696-2598182022ab\") " pod="kube-system/konnectivity-agent-cvsnb"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299888    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qxnhb\" (UniqueName: \"kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb\") pod \"network-check-target-hrcwh\" (UID: \"b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4\") " pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299915    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbl2w\" (UniqueName: \"kubernetes.io/projected/8b0b6f57-f7d5-494f-a47b-132adfbe518f-kube-api-access-wbl2w\") pod \"iptables-alerter-jpw2j\" (UID: \"8b0b6f57-f7d5-494f-a47b-132adfbe518f\") " pod="openshift-network-operator/iptables-alerter-jpw2j"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299931    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-sys-fs\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299947    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf00e197-64d4-4961-9730-248b6e1ec010-host\") pod \"node-ca-kmg5m\" (UID: \"cf00e197-64d4-4961-9730-248b6e1ec010\") " pod="openshift-image-registry/node-ca-kmg5m"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299964    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-slash\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.299990    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-run-ovn-kubernetes\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300008    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8b983f07-6127-487d-a077-b09e0d2c93f7-env-overrides\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300025    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300034    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-systemd-units\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300044    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-run-ovn\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.301443 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300111    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-multus-conf-dir\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300130    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8b0b6f57-f7d5-494f-a47b-132adfbe518f-iptables-alerter-script\") pod \"iptables-alerter-jpw2j\" (UID: \"8b0b6f57-f7d5-494f-a47b-132adfbe518f\") " pod="openshift-network-operator/iptables-alerter-jpw2j"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300152    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-host\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300227    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300270    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-cni-bin\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300304    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-cni-netd\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300350    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-registration-dir\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300406    2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300505    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-cni-binary-copy\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300709    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-kubelet\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300773    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4222bafb-9e5f-4884-832d-83364e16f3e9-hosts-file\") pod \"node-resolver-vc2ht\" (UID: \"4222bafb-9e5f-4884-832d-83364e16f3e9\") " pod="openshift-dns/node-resolver-vc2ht"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300825    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-device-dir\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.300882    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-etc-selinux\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.301084    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-system-cni-dir\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.301134    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.301276    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-run-ovn-kubernetes\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.301347    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf00e197-64d4-4961-9730-248b6e1ec010-host\") pod \"node-ca-kmg5m\" (UID: \"cf00e197-64d4-4961-9730-248b6e1ec010\") " pod="openshift-image-registry/node-ca-kmg5m"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.301366    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-host-slash\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.302006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.301371    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cf00e197-64d4-4961-9730-248b6e1ec010-serviceca\") pod \"node-ca-kmg5m\" (UID: \"cf00e197-64d4-4961-9730-248b6e1ec010\") " pod="openshift-image-registry/node-ca-kmg5m"
May 28 17:17:18.302741 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.301395    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-sys-fs\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.302741 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.301425    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-node-log\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.302741 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.301433    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-os-release\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.302741 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.301462    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8b983f07-6127-487d-a077-b09e0d2c93f7-run-systemd\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.302741 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.301785    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8b983f07-6127-487d-a077-b09e0d2c93f7-env-overrides\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.303660 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.303637    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b983f07-6127-487d-a077-b09e0d2c93f7-ovn-node-metrics-cert\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.304207 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.304177    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/737fe4ef-9e14-4558-8696-2598182022ab-agent-certs\") pod \"konnectivity-agent-cvsnb\" (UID: \"737fe4ef-9e14-4558-8696-2598182022ab\") " pod="kube-system/konnectivity-agent-cvsnb"
May 28 17:17:18.308119 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:18.307856    2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:17:18.308119 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:18.307878    2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:17:18.308119 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:18.307892    2561 projected.go:194] Error preparing data for projected volume kube-api-access-qxnhb for pod openshift-network-diagnostics/network-check-target-hrcwh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:17:18.308119 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:18.307949    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb podName:b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4 nodeName:}" failed. No retries permitted until 2026-05-28 17:17:18.807930959 +0000 UTC m=+3.047481333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qxnhb" (UniqueName: "kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb") pod "network-check-target-hrcwh" (UID: "b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:17:18.308559 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.308538    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njwl8\" (UniqueName: \"kubernetes.io/projected/4222bafb-9e5f-4884-832d-83364e16f3e9-kube-api-access-njwl8\") pod \"node-resolver-vc2ht\" (UID: \"4222bafb-9e5f-4884-832d-83364e16f3e9\") " pod="openshift-dns/node-resolver-vc2ht"
May 28 17:17:18.309527 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.309505    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpwcz\" (UniqueName: \"kubernetes.io/projected/cf00e197-64d4-4961-9730-248b6e1ec010-kube-api-access-rpwcz\") pod \"node-ca-kmg5m\" (UID: \"cf00e197-64d4-4961-9730-248b6e1ec010\") " pod="openshift-image-registry/node-ca-kmg5m"
May 28 17:17:18.309981 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.309959    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxmpn\" (UniqueName: \"kubernetes.io/projected/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-kube-api-access-vxmpn\") pod \"network-metrics-daemon-7xrml\" (UID: \"1a8e2081-5b89-4115-82b2-d1afbf6f58ed\") " pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:18.311257 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.311056    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwkdt\" (UniqueName: \"kubernetes.io/projected/4af98fa6-2b0c-4bdf-a453-414546d0cb0d-kube-api-access-cwkdt\") pod \"aws-ebs-csi-driver-node-8spfs\" (UID: \"4af98fa6-2b0c-4bdf-a453-414546d0cb0d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.312122 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.312096    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcdh4\" (UniqueName: \"kubernetes.io/projected/6ae61aa8-393c-4a9a-a9ae-f125c69e5871-kube-api-access-xcdh4\") pod \"multus-additional-cni-plugins-l55xd\" (UID: \"6ae61aa8-393c-4a9a-a9ae-f125c69e5871\") " pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.312576 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.312532    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvqk2\" (UniqueName: \"kubernetes.io/projected/8b983f07-6127-487d-a077-b09e0d2c93f7-kube-api-access-bvqk2\") pod \"ovnkube-node-5gxv8\" (UID: \"8b983f07-6127-487d-a077-b09e0d2c93f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.324497 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.324419    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" event={"ID":"312feb2e8872e44b4817c5bdff3cc0eb","Type":"ContainerStarted","Data":"cf6a4a972b6c6c44bcabfde5e7f1b9965062f55fb5c9037631433331b4441d2f"}
May 28 17:17:18.326665 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.326639    2561 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 17:17:18.328279 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.328244    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal" event={"ID":"8b6f5dedac2bb56dda3493221133ac3c","Type":"ContainerStarted","Data":"0c3a0d028654ee504f76e58ad7228a68946fc006a1e080fcc1dc1aa98fe12826"}
May 28 17:17:18.400859 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.400827    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-multus-conf-dir\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.400859 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.400863    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8b0b6f57-f7d5-494f-a47b-132adfbe518f-iptables-alerter-script\") pod \"iptables-alerter-jpw2j\" (UID: \"8b0b6f57-f7d5-494f-a47b-132adfbe518f\") " pod="openshift-network-operator/iptables-alerter-jpw2j"
May 28 17:17:18.401087 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.400889    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-host\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401087 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.400945    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-host\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401087 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.400960    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-multus-conf-dir\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401087 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401009    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-88mpc\" (UniqueName: \"kubernetes.io/projected/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-kube-api-access-88mpc\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401087 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401039    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-modprobe-d\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401087 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401064    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-system-cni-dir\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401087 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401088    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-multus-daemon-config\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401113    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-run-multus-certs\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401141    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b0b6f57-f7d5-494f-a47b-132adfbe518f-host-slash\") pod \"iptables-alerter-jpw2j\" (UID: \"8b0b6f57-f7d5-494f-a47b-132adfbe518f\") " pod="openshift-network-operator/iptables-alerter-jpw2j"
May 28 17:17:18.401410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401166    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-sysconfig\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401193    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-lib-modules\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401208    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-modprobe-d\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401226    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-run-multus-certs\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401227    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-os-release\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401268    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-var-lib-cni-bin\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401282    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-os-release\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401301    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-var-lib-kubelet\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401324    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-tuned\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401328    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b0b6f57-f7d5-494f-a47b-132adfbe518f-host-slash\") pod \"iptables-alerter-jpw2j\" (UID: \"8b0b6f57-f7d5-494f-a47b-132adfbe518f\") " pod="openshift-network-operator/iptables-alerter-jpw2j"
May 28 17:17:18.401410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401361    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-var-lib-cni-bin\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401375    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-sysconfig\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401389    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-sys\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401415    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-multus-socket-dir-parent\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401433    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-run-netns\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401448    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-systemd\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401191    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-system-cni-dir\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401461    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-lib-modules\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401491    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8b0b6f57-f7d5-494f-a47b-132adfbe518f-iptables-alerter-script\") pod \"iptables-alerter-jpw2j\" (UID: \"8b0b6f57-f7d5-494f-a47b-132adfbe518f\") " pod="openshift-network-operator/iptables-alerter-jpw2j"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401503    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-multus-socket-dir-parent\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401509    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-cnibin\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401538    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-systemd\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401545    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-sysctl-d\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401552    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-cnibin\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401535    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-sys\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401539    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-var-lib-kubelet\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401598    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-run-netns\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401642    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-sysctl-d\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401670    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-sysctl-conf\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401719    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-run\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401744    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-snshj\" (UniqueName: \"kubernetes.io/projected/00d2dbe6-3f97-4fab-8af5-581a8329be5c-kube-api-access-snshj\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.401994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401773    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-hostroot\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401794    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-run\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401801    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-run-k8s-cni-cncf-io\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401820    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-sysctl-conf\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401825    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-var-lib-cni-multus\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401849    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d2dbe6-3f97-4fab-8af5-581a8329be5c-tmp\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401866    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-hostroot\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401874    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-etc-kubernetes\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401897    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-var-lib-kubelet\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401958    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-run-k8s-cni-cncf-io\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401996    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-etc-kubernetes\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.402110    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-var-lib-kubelet\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.402210    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-multus-cni-dir\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.402238    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-cni-binary-copy\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.402260    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-kubernetes\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.402266    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-multus-daemon-config\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.402308    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wbl2w\" (UniqueName: \"kubernetes.io/projected/8b0b6f57-f7d5-494f-a47b-132adfbe518f-kube-api-access-wbl2w\") pod \"iptables-alerter-jpw2j\" (UID: \"8b0b6f57-f7d5-494f-a47b-132adfbe518f\") " pod="openshift-network-operator/iptables-alerter-jpw2j"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.402325    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-kubernetes\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.402658 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.401898    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-host-var-lib-cni-multus\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.403563 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.402560    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-multus-cni-dir\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.403563 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.402892    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-cni-binary-copy\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.403866 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.403838    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/00d2dbe6-3f97-4fab-8af5-581a8329be5c-etc-tuned\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.404109 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.404092    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d2dbe6-3f97-4fab-8af5-581a8329be5c-tmp\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.410287 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.410248    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88mpc\" (UniqueName: \"kubernetes.io/projected/b11ba88d-88e2-40a4-8cef-be08cd1b3f47-kube-api-access-88mpc\") pod \"multus-h7tx9\" (UID: \"b11ba88d-88e2-40a4-8cef-be08cd1b3f47\") " pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.410769 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.410751    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbl2w\" (UniqueName: \"kubernetes.io/projected/8b0b6f57-f7d5-494f-a47b-132adfbe518f-kube-api-access-wbl2w\") pod \"iptables-alerter-jpw2j\" (UID: \"8b0b6f57-f7d5-494f-a47b-132adfbe518f\") " pod="openshift-network-operator/iptables-alerter-jpw2j"
May 28 17:17:18.411258 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.411234    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snshj\" (UniqueName: \"kubernetes.io/projected/00d2dbe6-3f97-4fab-8af5-581a8329be5c-kube-api-access-snshj\") pod \"tuned-cj8n6\" (UID: \"00d2dbe6-3f97-4fab-8af5-581a8329be5c\") " pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.492444 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.492319    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs"
May 28 17:17:18.501165 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.501141    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:18.509872 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.509842    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cvsnb"
May 28 17:17:18.516569 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.516545    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vc2ht"
May 28 17:17:18.524235 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.524208    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kmg5m"
May 28 17:17:18.531552 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.531523    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l55xd"
May 28 17:17:18.537470 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.537451    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h7tx9"
May 28 17:17:18.545240 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.545216    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jpw2j"
May 28 17:17:18.546005 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.545720    2561 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 17:17:18.550119 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.550101    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cj8n6"
May 28 17:17:18.805248 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.805179    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs\") pod \"network-metrics-daemon-7xrml\" (UID: \"1a8e2081-5b89-4115-82b2-d1afbf6f58ed\") " pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:18.805377 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:18.805303    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:17:18.805414 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:18.805405    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs podName:1a8e2081-5b89-4115-82b2-d1afbf6f58ed nodeName:}" failed. No retries permitted until 2026-05-28 17:17:19.805385666 +0000 UTC m=+4.044936042 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs") pod "network-metrics-daemon-7xrml" (UID: "1a8e2081-5b89-4115-82b2-d1afbf6f58ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:17:18.905563 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:18.905525    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qxnhb\" (UniqueName: \"kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb\") pod \"network-check-target-hrcwh\" (UID: \"b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4\") " pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:18.905815 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:18.905648    2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:17:18.905815 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:18.905716    2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:17:18.905815 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:18.905729    2561 projected.go:194] Error preparing data for projected volume kube-api-access-qxnhb for pod openshift-network-diagnostics/network-check-target-hrcwh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:17:18.905815 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:18.905786    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb podName:b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4 nodeName:}" failed. No retries permitted until 2026-05-28 17:17:19.905764805 +0000 UTC m=+4.145315193 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxnhb" (UniqueName: "kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb") pod "network-check-target-hrcwh" (UID: "b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:17:19.160651 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:19.160624    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b0b6f57_f7d5_494f_a47b_132adfbe518f.slice/crio-8081030d2de1831ce5177e7550deecad3238e94aca4d976ebcd8027d5c8454fd WatchSource:0}: Error finding container 8081030d2de1831ce5177e7550deecad3238e94aca4d976ebcd8027d5c8454fd: Status 404 returned error can't find the container with id 8081030d2de1831ce5177e7550deecad3238e94aca4d976ebcd8027d5c8454fd
May 28 17:17:19.162126 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:19.161939    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod737fe4ef_9e14_4558_8696_2598182022ab.slice/crio-129113a7ef41a792ee0396bf9a9a0ad396951d663a4fd3d7f17726389512e640 WatchSource:0}: Error finding container 129113a7ef41a792ee0396bf9a9a0ad396951d663a4fd3d7f17726389512e640: Status 404 returned error can't find the container with id 129113a7ef41a792ee0396bf9a9a0ad396951d663a4fd3d7f17726389512e640
May 28 17:17:19.163190 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:19.163154    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae61aa8_393c_4a9a_a9ae_f125c69e5871.slice/crio-8868c26c92d742fce88727d7a29b9fa0586861a83ee974c84848f68e6323648f WatchSource:0}: Error finding container 8868c26c92d742fce88727d7a29b9fa0586861a83ee974c84848f68e6323648f: Status 404 returned error can't find the container with id 8868c26c92d742fce88727d7a29b9fa0586861a83ee974c84848f68e6323648f
May 28 17:17:19.165097 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:19.164971    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4af98fa6_2b0c_4bdf_a453_414546d0cb0d.slice/crio-31b374c42afe3be7d009db0d0f75f9977bab826c767c1bdb138b2b669c46591c WatchSource:0}: Error finding container 31b374c42afe3be7d009db0d0f75f9977bab826c767c1bdb138b2b669c46591c: Status 404 returned error can't find the container with id 31b374c42afe3be7d009db0d0f75f9977bab826c767c1bdb138b2b669c46591c
May 28 17:17:19.166240 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:19.166216    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb11ba88d_88e2_40a4_8cef_be08cd1b3f47.slice/crio-8f187aee579112ce699d382da09f4a2325cbe4cf609899435753a4de267d4bd1 WatchSource:0}: Error finding container 8f187aee579112ce699d382da09f4a2325cbe4cf609899435753a4de267d4bd1: Status 404 returned error can't find the container with id 8f187aee579112ce699d382da09f4a2325cbe4cf609899435753a4de267d4bd1
May 28 17:17:19.167013 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:19.166990    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00d2dbe6_3f97_4fab_8af5_581a8329be5c.slice/crio-038a4225a82ff77629a2be24b2bf7dd1314cd7ff67e5282005ed8f0553ccb41d WatchSource:0}: Error finding container 038a4225a82ff77629a2be24b2bf7dd1314cd7ff67e5282005ed8f0553ccb41d: Status 404 returned error can't find the container with id 038a4225a82ff77629a2be24b2bf7dd1314cd7ff67e5282005ed8f0553ccb41d
May 28 17:17:19.168468 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:19.168270    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b983f07_6127_487d_a077_b09e0d2c93f7.slice/crio-047782012cbc9e93595b537c9c989536c3b5ea78a8c0fb82342af985d18c5ecf WatchSource:0}: Error finding container 047782012cbc9e93595b537c9c989536c3b5ea78a8c0fb82342af985d18c5ecf: Status 404 returned error can't find the container with id 047782012cbc9e93595b537c9c989536c3b5ea78a8c0fb82342af985d18c5ecf
May 28 17:17:19.169904 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:19.169791    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf00e197_64d4_4961_9730_248b6e1ec010.slice/crio-91ab1f823859aee40eb00503bd0433d662a8976068db3d2db92dc55517cc54f9 WatchSource:0}: Error finding container 91ab1f823859aee40eb00503bd0433d662a8976068db3d2db92dc55517cc54f9: Status 404 returned error can't find the container with id 91ab1f823859aee40eb00503bd0433d662a8976068db3d2db92dc55517cc54f9
May 28 17:17:19.170474 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:19.170341    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4222bafb_9e5f_4884_832d_83364e16f3e9.slice/crio-b20ee5caf37f3db355d7a706b82a6663801003e47a0f02d44586a4873315f458 WatchSource:0}: Error finding container b20ee5caf37f3db355d7a706b82a6663801003e47a0f02d44586a4873315f458: Status 404 returned error can't find the container with id b20ee5caf37f3db355d7a706b82a6663801003e47a0f02d44586a4873315f458
May 28 17:17:19.228995 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:19.228967    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 17:12:17 +0000 UTC" deadline="2028-01-10 15:02:41.688434087 +0000 UTC"
May 28 17:17:19.228995 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:19.228991    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14205h45m22.45944508s"
May 28 17:17:19.316923 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:19.316887    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:19.317050 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:19.317009    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:19.331690 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:19.331643    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h7tx9" event={"ID":"b11ba88d-88e2-40a4-8cef-be08cd1b3f47","Type":"ContainerStarted","Data":"8f187aee579112ce699d382da09f4a2325cbe4cf609899435753a4de267d4bd1"}
May 28 17:17:19.334405 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:19.334381    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jpw2j" event={"ID":"8b0b6f57-f7d5-494f-a47b-132adfbe518f","Type":"ContainerStarted","Data":"8081030d2de1831ce5177e7550deecad3238e94aca4d976ebcd8027d5c8454fd"}
May 28 17:17:19.337301 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:19.337275    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vc2ht" event={"ID":"4222bafb-9e5f-4884-832d-83364e16f3e9","Type":"ContainerStarted","Data":"b20ee5caf37f3db355d7a706b82a6663801003e47a0f02d44586a4873315f458"}
May 28 17:17:19.338226 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:19.338208    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kmg5m" event={"ID":"cf00e197-64d4-4961-9730-248b6e1ec010","Type":"ContainerStarted","Data":"91ab1f823859aee40eb00503bd0433d662a8976068db3d2db92dc55517cc54f9"}
May 28 17:17:19.339116 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:19.339099    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cj8n6" event={"ID":"00d2dbe6-3f97-4fab-8af5-581a8329be5c","Type":"ContainerStarted","Data":"038a4225a82ff77629a2be24b2bf7dd1314cd7ff67e5282005ed8f0553ccb41d"}
May 28 17:17:19.340055 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:19.340036    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs" event={"ID":"4af98fa6-2b0c-4bdf-a453-414546d0cb0d","Type":"ContainerStarted","Data":"31b374c42afe3be7d009db0d0f75f9977bab826c767c1bdb138b2b669c46591c"}
May 28 17:17:19.342227 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:19.342202    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cvsnb" event={"ID":"737fe4ef-9e14-4558-8696-2598182022ab","Type":"ContainerStarted","Data":"129113a7ef41a792ee0396bf9a9a0ad396951d663a4fd3d7f17726389512e640"}
May 28 17:17:19.343660 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:19.343641    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l55xd" event={"ID":"6ae61aa8-393c-4a9a-a9ae-f125c69e5871","Type":"ContainerStarted","Data":"8868c26c92d742fce88727d7a29b9fa0586861a83ee974c84848f68e6323648f"}
May 28 17:17:19.347432 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:19.347410    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal" event={"ID":"8b6f5dedac2bb56dda3493221133ac3c","Type":"ContainerStarted","Data":"c8b64180ab7db7cd96e3fc41c33d1e70549669a89fcfa1daa166a802b21e72a6"}
May 28 17:17:19.349269 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:19.349245    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8" event={"ID":"8b983f07-6127-487d-a077-b09e0d2c93f7","Type":"ContainerStarted","Data":"047782012cbc9e93595b537c9c989536c3b5ea78a8c0fb82342af985d18c5ecf"}
May 28 17:17:19.362657 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:19.362617    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal" podStartSLOduration=2.36260761 podStartE2EDuration="2.36260761s" podCreationTimestamp="2026-05-28 17:17:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:17:19.362448309 +0000 UTC m=+3.601998699" watchObservedRunningTime="2026-05-28 17:17:19.36260761 +0000 UTC m=+3.602157999"
May 28 17:17:19.811583 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:19.811549    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs\") pod \"network-metrics-daemon-7xrml\" (UID: \"1a8e2081-5b89-4115-82b2-d1afbf6f58ed\") " pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:19.811758 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:19.811732    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:17:19.811847 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:19.811797    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs podName:1a8e2081-5b89-4115-82b2-d1afbf6f58ed nodeName:}" failed. No retries permitted until 2026-05-28 17:17:21.81177699 +0000 UTC m=+6.051327362 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs") pod "network-metrics-daemon-7xrml" (UID: "1a8e2081-5b89-4115-82b2-d1afbf6f58ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:17:19.913267 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:19.912447    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qxnhb\" (UniqueName: \"kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb\") pod \"network-check-target-hrcwh\" (UID: \"b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4\") " pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:19.913267 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:19.912601    2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:17:19.913267 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:19.912620    2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:17:19.913267 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:19.912633    2561 projected.go:194] Error preparing data for projected volume kube-api-access-qxnhb for pod openshift-network-diagnostics/network-check-target-hrcwh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:17:19.913267 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:19.912709    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb podName:b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4 nodeName:}" failed. No retries permitted until 2026-05-28 17:17:21.912670922 +0000 UTC m=+6.152221303 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxnhb" (UniqueName: "kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb") pod "network-check-target-hrcwh" (UID: "b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:17:20.318391 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:20.317909    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:20.318391 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:20.318043    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:20.357977 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:20.357932    2561 generic.go:358] "Generic (PLEG): container finished" podID="312feb2e8872e44b4817c5bdff3cc0eb" containerID="549c7fcb74977b5a25491bc3ca39b8ced5ce5b395393927871445739e10da94a" exitCode=0
May 28 17:17:20.358864 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:20.358836    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" event={"ID":"312feb2e8872e44b4817c5bdff3cc0eb","Type":"ContainerDied","Data":"549c7fcb74977b5a25491bc3ca39b8ced5ce5b395393927871445739e10da94a"}
May 28 17:17:21.317838 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:21.317728    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:21.318024 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:21.317909    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:21.371448 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:21.371358    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" event={"ID":"312feb2e8872e44b4817c5bdff3cc0eb","Type":"ContainerStarted","Data":"dc1f42480f2bd0f38b7f9c33bfb50120314eed7c5621abad1a286fb14704a28a"}
May 28 17:17:21.390306 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:21.389829    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" podStartSLOduration=4.389808876 podStartE2EDuration="4.389808876s" podCreationTimestamp="2026-05-28 17:17:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:17:21.388998113 +0000 UTC m=+5.628548504" watchObservedRunningTime="2026-05-28 17:17:21.389808876 +0000 UTC m=+5.629359267"
May 28 17:17:21.828376 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:21.828336    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs\") pod \"network-metrics-daemon-7xrml\" (UID: \"1a8e2081-5b89-4115-82b2-d1afbf6f58ed\") " pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:21.828533 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:21.828497    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:17:21.828582 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:21.828564    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs podName:1a8e2081-5b89-4115-82b2-d1afbf6f58ed nodeName:}" failed. No retries permitted until 2026-05-28 17:17:25.828544528 +0000 UTC m=+10.068094910 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs") pod "network-metrics-daemon-7xrml" (UID: "1a8e2081-5b89-4115-82b2-d1afbf6f58ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:17:21.929757 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:21.929136    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qxnhb\" (UniqueName: \"kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb\") pod \"network-check-target-hrcwh\" (UID: \"b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4\") " pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:21.929757 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:21.929294    2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:17:21.929757 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:21.929313    2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:17:21.929757 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:21.929328    2561 projected.go:194] Error preparing data for projected volume kube-api-access-qxnhb for pod openshift-network-diagnostics/network-check-target-hrcwh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:17:21.929757 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:21.929390    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb podName:b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4 nodeName:}" failed. No retries permitted until 2026-05-28 17:17:25.929370558 +0000 UTC m=+10.168920934 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxnhb" (UniqueName: "kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb") pod "network-check-target-hrcwh" (UID: "b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:17:22.318912 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:22.318874    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:22.319078 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:22.319012    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:23.317587 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:23.317531    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:23.318044 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:23.317641    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:24.317699 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:24.317642    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:24.318121 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:24.317825    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:25.318050 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:25.317529    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:25.318050 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:25.317654    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:25.863026 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:25.862988    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs\") pod \"network-metrics-daemon-7xrml\" (UID: \"1a8e2081-5b89-4115-82b2-d1afbf6f58ed\") " pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:25.863214 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:25.863170    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:17:25.863260 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:25.863230    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs podName:1a8e2081-5b89-4115-82b2-d1afbf6f58ed nodeName:}" failed. No retries permitted until 2026-05-28 17:17:33.863216594 +0000 UTC m=+18.102766968 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs") pod "network-metrics-daemon-7xrml" (UID: "1a8e2081-5b89-4115-82b2-d1afbf6f58ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:17:25.964063 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:25.964029    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qxnhb\" (UniqueName: \"kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb\") pod \"network-check-target-hrcwh\" (UID: \"b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4\") " pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:25.964233 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:25.964191    2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:17:25.964233 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:25.964213    2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:17:25.964233 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:25.964227    2561 projected.go:194] Error preparing data for projected volume kube-api-access-qxnhb for pod openshift-network-diagnostics/network-check-target-hrcwh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:17:25.964383 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:25.964276    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb podName:b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4 nodeName:}" failed. No retries permitted until 2026-05-28 17:17:33.964261769 +0000 UTC m=+18.203812137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxnhb" (UniqueName: "kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb") pod "network-check-target-hrcwh" (UID: "b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:17:26.319165 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:26.318737    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:26.319165 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:26.318883    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:27.317607 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:27.317199    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:27.317607 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:27.317307    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:28.318293 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:28.317774    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:28.318293 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:28.317916    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:29.317356 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:29.317320    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:29.317503 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:29.317424    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:30.317489 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:30.317450    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:30.317983 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:30.317571    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:31.317348 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:31.317310    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:31.317506 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:31.317431    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:32.317584 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:32.317546    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:32.318009 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:32.317707    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:33.316730 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:33.316653    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:33.316935 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:33.316841    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:33.921769 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:33.921735    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs\") pod \"network-metrics-daemon-7xrml\" (UID: \"1a8e2081-5b89-4115-82b2-d1afbf6f58ed\") " pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:33.922184 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:33.921909    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:17:33.922184 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:33.921981    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs podName:1a8e2081-5b89-4115-82b2-d1afbf6f58ed nodeName:}" failed. No retries permitted until 2026-05-28 17:17:49.921964687 +0000 UTC m=+34.161515058 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs") pod "network-metrics-daemon-7xrml" (UID: "1a8e2081-5b89-4115-82b2-d1afbf6f58ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:17:34.022485 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:34.022444    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qxnhb\" (UniqueName: \"kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb\") pod \"network-check-target-hrcwh\" (UID: \"b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4\") " pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:34.022649 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:34.022610    2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:17:34.022649 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:34.022633    2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:17:34.022649 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:34.022648    2561 projected.go:194] Error preparing data for projected volume kube-api-access-qxnhb for pod openshift-network-diagnostics/network-check-target-hrcwh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:17:34.022803 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:34.022733    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb podName:b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4 nodeName:}" failed. No retries permitted until 2026-05-28 17:17:50.022714255 +0000 UTC m=+34.262264625 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxnhb" (UniqueName: "kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb") pod "network-check-target-hrcwh" (UID: "b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:17:34.316940 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:34.316900    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:34.317089 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:34.317028    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:35.317293 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:35.317260    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:35.317766 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:35.317365    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:36.317566 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:36.317335    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:36.318054 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:36.317655    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:37.317300 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:37.317267    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:37.317455 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:37.317368    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:38.317768 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.317503    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:38.318750 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:38.317888    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:38.402262 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.402182    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs" event={"ID":"4af98fa6-2b0c-4bdf-a453-414546d0cb0d","Type":"ContainerStarted","Data":"564e9f22479ad32db8d9415001eb0d0a3204ba4562d73cc500ddf6000151e8fb"}
May 28 17:17:38.403447 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.403419    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cvsnb" event={"ID":"737fe4ef-9e14-4558-8696-2598182022ab","Type":"ContainerStarted","Data":"8135d5228e874e209efd443642fb315cac14306e7536dfb59e54e46630b9670c"}
May 28 17:17:38.404943 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.404915    2561 generic.go:358] "Generic (PLEG): container finished" podID="6ae61aa8-393c-4a9a-a9ae-f125c69e5871" containerID="97e4f5659224c9f2f37b46d868df327c6f68034b079081719f7aee242bfd128b" exitCode=0
May 28 17:17:38.405029 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.405007    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l55xd" event={"ID":"6ae61aa8-393c-4a9a-a9ae-f125c69e5871","Type":"ContainerDied","Data":"97e4f5659224c9f2f37b46d868df327c6f68034b079081719f7aee242bfd128b"}
May 28 17:17:38.408290 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.408259    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8" event={"ID":"8b983f07-6127-487d-a077-b09e0d2c93f7","Type":"ContainerStarted","Data":"6a2063133a5b267b538ed1a32167ccf0216d03b30fbd0e0937cc7bf71573e5fe"}
May 28 17:17:38.408385 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.408292    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8" event={"ID":"8b983f07-6127-487d-a077-b09e0d2c93f7","Type":"ContainerStarted","Data":"50a5605a07d7f3497d6c1606f52ee9389d34fddcca3fdf398516cff4ac331c43"}
May 28 17:17:38.408385 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.408307    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8" event={"ID":"8b983f07-6127-487d-a077-b09e0d2c93f7","Type":"ContainerStarted","Data":"ac38758f65d9324e2da7e31a8bcb44ad0c497097470ecf5d3c44a96d31b4df22"}
May 28 17:17:38.408385 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.408320    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8" event={"ID":"8b983f07-6127-487d-a077-b09e0d2c93f7","Type":"ContainerStarted","Data":"35f184011328216b4d786d7176d8d3bef4f646bdab6bb2294fd5221d3e5fac24"}
May 28 17:17:38.408385 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.408333    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8" event={"ID":"8b983f07-6127-487d-a077-b09e0d2c93f7","Type":"ContainerStarted","Data":"a3e9cddc7b78e4762035d7c16ef0f964ffdf321c2e752f64d836cedeba0a9968"}
May 28 17:17:38.408385 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.408345    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8" event={"ID":"8b983f07-6127-487d-a077-b09e0d2c93f7","Type":"ContainerStarted","Data":"b88ada94081ef700f7c4358d7121d75e8e41a5ed1c34e7ce94f7e6d6cfa234e6"}
May 28 17:17:38.409698 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.409659    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h7tx9" event={"ID":"b11ba88d-88e2-40a4-8cef-be08cd1b3f47","Type":"ContainerStarted","Data":"cc380de3ce64e7a6271f62abfaf902bb578960c033eefb04511158a89259af03"}
May 28 17:17:38.410917 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.410898    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vc2ht" event={"ID":"4222bafb-9e5f-4884-832d-83364e16f3e9","Type":"ContainerStarted","Data":"4f7afd3d5ab5f3232e7558f528ebf1c7d631dd83cb7762d6d58509276b4c1502"}
May 28 17:17:38.412500 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.412478    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kmg5m" event={"ID":"cf00e197-64d4-4961-9730-248b6e1ec010","Type":"ContainerStarted","Data":"2559a5ac90d8a1887cca79e4bf520285c898f4a6a0bb317d7e82987acfadb287"}
May 28 17:17:38.415352 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.415328    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cj8n6" event={"ID":"00d2dbe6-3f97-4fab-8af5-581a8329be5c","Type":"ContainerStarted","Data":"a8f76afeaef26fbc10da080ea3caae5c0606360cf5a68cc203e80ac7d4fc2e2d"}
May 28 17:17:38.419521 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.419471    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cvsnb" podStartSLOduration=4.02370625 podStartE2EDuration="22.419455949s" podCreationTimestamp="2026-05-28 17:17:16 +0000 UTC" firstStartedPulling="2026-05-28 17:17:19.164106156 +0000 UTC m=+3.403656531" lastFinishedPulling="2026-05-28 17:17:37.559855847 +0000 UTC m=+21.799406230" observedRunningTime="2026-05-28 17:17:38.418813972 +0000 UTC m=+22.658364367" watchObservedRunningTime="2026-05-28 17:17:38.419455949 +0000 UTC m=+22.659006340"
May 28 17:17:38.439411 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.439305    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-vc2ht" podStartSLOduration=8.860250523 podStartE2EDuration="22.439286117s" podCreationTimestamp="2026-05-28 17:17:16 +0000 UTC" firstStartedPulling="2026-05-28 17:17:19.172626241 +0000 UTC m=+3.412176609" lastFinishedPulling="2026-05-28 17:17:32.751661835 +0000 UTC m=+16.991212203" observedRunningTime="2026-05-28 17:17:38.438778312 +0000 UTC m=+22.678328694" watchObservedRunningTime="2026-05-28 17:17:38.439286117 +0000 UTC m=+22.678836508"
May 28 17:17:38.455395 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.455349    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kmg5m" podStartSLOduration=8.875594319 podStartE2EDuration="22.455335331s" podCreationTimestamp="2026-05-28 17:17:16 +0000 UTC" firstStartedPulling="2026-05-28 17:17:19.171917268 +0000 UTC m=+3.411467635" lastFinishedPulling="2026-05-28 17:17:32.751658266 +0000 UTC m=+16.991208647" observedRunningTime="2026-05-28 17:17:38.455275892 +0000 UTC m=+22.694826282" watchObservedRunningTime="2026-05-28 17:17:38.455335331 +0000 UTC m=+22.694885721"
May 28 17:17:38.511197 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.510989    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cj8n6" podStartSLOduration=4.119669373 podStartE2EDuration="22.51097214s" podCreationTimestamp="2026-05-28 17:17:16 +0000 UTC" firstStartedPulling="2026-05-28 17:17:19.16858048 +0000 UTC m=+3.408130851" lastFinishedPulling="2026-05-28 17:17:37.559883234 +0000 UTC m=+21.799433618" observedRunningTime="2026-05-28 17:17:38.51086862 +0000 UTC m=+22.750419010" watchObservedRunningTime="2026-05-28 17:17:38.51097214 +0000 UTC m=+22.750522527"
May 28 17:17:38.511811 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.511773    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-h7tx9" podStartSLOduration=4.101260998 podStartE2EDuration="22.511764731s" podCreationTimestamp="2026-05-28 17:17:16 +0000 UTC" firstStartedPulling="2026-05-28 17:17:19.167873603 +0000 UTC m=+3.407423970" lastFinishedPulling="2026-05-28 17:17:37.578377332 +0000 UTC m=+21.817927703" observedRunningTime="2026-05-28 17:17:38.492536494 +0000 UTC m=+22.732086883" watchObservedRunningTime="2026-05-28 17:17:38.511764731 +0000 UTC m=+22.751315159"
May 28 17:17:38.744353 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:38.744330    2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 28 17:17:39.258936 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:39.258747    2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T17:17:38.744348884Z","UUID":"447a1d03-1921-495b-88b9-a1c68b9e791b","Handler":null,"Name":"","Endpoint":""}
May 28 17:17:39.260737 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:39.260705    2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 28 17:17:39.260859 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:39.260748    2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 28 17:17:39.317635 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:39.317611    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:39.317763 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:39.317737    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:39.419250 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:39.419214    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs" event={"ID":"4af98fa6-2b0c-4bdf-a453-414546d0cb0d","Type":"ContainerStarted","Data":"3d65fc8aeaf17ac3bbe3c15b499b87973093ed9b8bd4255c8dabaac001db9592"}
May 28 17:17:39.421313 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:39.421277    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jpw2j" event={"ID":"8b0b6f57-f7d5-494f-a47b-132adfbe518f","Type":"ContainerStarted","Data":"bba32385daa7a2367b8783ea80f9b0cd80cfeb083d41a4d6c81627988e949513"}
May 28 17:17:39.446773 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:39.446708    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jpw2j" podStartSLOduration=5.048521829 podStartE2EDuration="23.446670817s" podCreationTimestamp="2026-05-28 17:17:16 +0000 UTC" firstStartedPulling="2026-05-28 17:17:19.162201491 +0000 UTC m=+3.401751860" lastFinishedPulling="2026-05-28 17:17:37.560350481 +0000 UTC m=+21.799900848" observedRunningTime="2026-05-28 17:17:39.446367483 +0000 UTC m=+23.685917879" watchObservedRunningTime="2026-05-28 17:17:39.446670817 +0000 UTC m=+23.686221209"
May 28 17:17:39.725035 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:39.724998    2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cvsnb"
May 28 17:17:39.725684 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:39.725658    2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cvsnb"
May 28 17:17:40.317532 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:40.317498    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:40.317701 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:40.317644    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:40.427046 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:40.427015    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8" event={"ID":"8b983f07-6127-487d-a077-b09e0d2c93f7","Type":"ContainerStarted","Data":"7750d97365cf96e67bb1c6247a8066ceb05b4a526e9643a5f73ad028b2b46c73"}
May 28 17:17:40.430066 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:40.429099    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs" event={"ID":"4af98fa6-2b0c-4bdf-a453-414546d0cb0d","Type":"ContainerStarted","Data":"2a96d8bcc453e96c4d732baef8482594370ff3ad9c5aa74104386e9450e341dd"}
May 28 17:17:40.430066 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:40.429468    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cvsnb"
May 28 17:17:40.430253 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:40.430208    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cvsnb"
May 28 17:17:40.448234 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:40.448180    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8spfs" podStartSLOduration=4.039336353 podStartE2EDuration="24.448164481s" podCreationTimestamp="2026-05-28 17:17:16 +0000 UTC" firstStartedPulling="2026-05-28 17:17:19.166626939 +0000 UTC m=+3.406177308" lastFinishedPulling="2026-05-28 17:17:39.575455066 +0000 UTC m=+23.815005436" observedRunningTime="2026-05-28 17:17:40.447929517 +0000 UTC m=+24.687479940" watchObservedRunningTime="2026-05-28 17:17:40.448164481 +0000 UTC m=+24.687714872"
May 28 17:17:41.316995 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:41.316963    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:41.317190 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:41.317084    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:42.317460 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:42.317426    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:42.318109 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:42.317536    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:43.317815 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:43.317624    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:43.318227 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:43.317898    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:43.435362 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:43.435332    2561 generic.go:358] "Generic (PLEG): container finished" podID="6ae61aa8-393c-4a9a-a9ae-f125c69e5871" containerID="3524d0d9387e898152583d6f4edd3c9520d39c37142f6baf3f291464d6d9459c" exitCode=0
May 28 17:17:43.435519 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:43.435418    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l55xd" event={"ID":"6ae61aa8-393c-4a9a-a9ae-f125c69e5871","Type":"ContainerDied","Data":"3524d0d9387e898152583d6f4edd3c9520d39c37142f6baf3f291464d6d9459c"}
May 28 17:17:43.438664 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:43.438644    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8" event={"ID":"8b983f07-6127-487d-a077-b09e0d2c93f7","Type":"ContainerStarted","Data":"4cc503b6b0f5a0e6587224a3f5299c2bcb40e86b2c104ff277de4c3fba3db5e3"}
May 28 17:17:43.439017 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:43.438992    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:43.439017 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:43.439018    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:43.439147 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:43.439028    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:43.454851 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:43.454830    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:43.454965 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:43.454893    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:17:43.485563 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:43.485517    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8" podStartSLOduration=9.047455757 podStartE2EDuration="27.485504332s" podCreationTimestamp="2026-05-28 17:17:16 +0000 UTC" firstStartedPulling="2026-05-28 17:17:19.170860837 +0000 UTC m=+3.410411205" lastFinishedPulling="2026-05-28 17:17:37.608909399 +0000 UTC m=+21.848459780" observedRunningTime="2026-05-28 17:17:43.485061932 +0000 UTC m=+27.724612321" watchObservedRunningTime="2026-05-28 17:17:43.485504332 +0000 UTC m=+27.725054719"
May 28 17:17:44.317434 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:44.317267    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:44.317581 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:44.317515    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:44.471973 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:44.471948    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hrcwh"]
May 28 17:17:44.472289 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:44.472073    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:44.472289 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:44.472156    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:44.474449 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:44.474425    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7xrml"]
May 28 17:17:44.474549 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:44.474503    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:44.474603 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:44.474584    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:45.444217 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:45.444187    2561 generic.go:358] "Generic (PLEG): container finished" podID="6ae61aa8-393c-4a9a-a9ae-f125c69e5871" containerID="20f0fce67f0332e21346e1e46f1099b2bc9090502ae182dd3d98f6cb2a99d72c" exitCode=0
May 28 17:17:45.444395 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:45.444275    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l55xd" event={"ID":"6ae61aa8-393c-4a9a-a9ae-f125c69e5871","Type":"ContainerDied","Data":"20f0fce67f0332e21346e1e46f1099b2bc9090502ae182dd3d98f6cb2a99d72c"}
May 28 17:17:46.317515 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:46.317469    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:46.318049 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:46.317570    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:46.318049 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:46.317652    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:46.318049 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:46.317770    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:47.450006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:47.449909    2561 generic.go:358] "Generic (PLEG): container finished" podID="6ae61aa8-393c-4a9a-a9ae-f125c69e5871" containerID="f9e7a0b9e279e08ccf5f68bcc957958f3b4e1ed22d19de7f210dac553304ea7e" exitCode=0
May 28 17:17:47.450006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:47.449974    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l55xd" event={"ID":"6ae61aa8-393c-4a9a-a9ae-f125c69e5871","Type":"ContainerDied","Data":"f9e7a0b9e279e08ccf5f68bcc957958f3b4e1ed22d19de7f210dac553304ea7e"}
May 28 17:17:48.317208 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:48.316744    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:48.317208 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:48.316876    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:48.317208 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:48.316936    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:48.317208 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:48.317033    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:49.935368 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:49.935336    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs\") pod \"network-metrics-daemon-7xrml\" (UID: \"1a8e2081-5b89-4115-82b2-d1afbf6f58ed\") " pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:49.935818 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:49.935475    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:17:49.935818 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:49.935532    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs podName:1a8e2081-5b89-4115-82b2-d1afbf6f58ed nodeName:}" failed. No retries permitted until 2026-05-28 17:18:21.935516394 +0000 UTC m=+66.175066766 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs") pod "network-metrics-daemon-7xrml" (UID: "1a8e2081-5b89-4115-82b2-d1afbf6f58ed") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:17:50.036315 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.036277    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qxnhb\" (UniqueName: \"kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb\") pod \"network-check-target-hrcwh\" (UID: \"b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4\") " pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:50.036473 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:50.036423    2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:17:50.036473 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:50.036440    2561 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:17:50.036473 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:50.036453    2561 projected.go:194] Error preparing data for projected volume kube-api-access-qxnhb for pod openshift-network-diagnostics/network-check-target-hrcwh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:17:50.036583 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:50.036516    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb podName:b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4 nodeName:}" failed. No retries permitted until 2026-05-28 17:18:22.036497146 +0000 UTC m=+66.276047521 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxnhb" (UniqueName: "kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb") pod "network-check-target-hrcwh" (UID: "b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:17:50.318060 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.317567    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:50.318060 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.317606    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:50.318060 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:50.317745    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xrml" podUID="1a8e2081-5b89-4115-82b2-d1afbf6f58ed"
May 28 17:17:50.318060 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:50.317869    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrcwh" podUID="b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4"
May 28 17:17:50.633152 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.633078    2561 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeReady"
May 28 17:17:50.633284 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.633215    2561 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 28 17:17:50.698752 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.698703    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jn46x"]
May 28 17:17:50.720902 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.720872    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9mdhg"]
May 28 17:17:50.721094 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.721067    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jn46x"
May 28 17:17:50.723827 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.723799    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 28 17:17:50.724117 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.724100    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 28 17:17:50.724316 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.724302    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sh6sk\""
May 28 17:17:50.736973 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.736949    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jn46x"]
May 28 17:17:50.737072 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.736981    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9mdhg"]
May 28 17:17:50.737072 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.737050    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9mdhg"
May 28 17:17:50.740000 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.739829    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 28 17:17:50.740000 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.739900    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 28 17:17:50.740201 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.740183    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 28 17:17:50.740427 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.740407    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nznc5\""
May 28 17:17:50.844879 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.842201    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:17:50.844879 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.842534    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bac47327-8e2e-410d-9a4b-53d26f170d1b-config-volume\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:17:50.844879 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.842593    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nxz8\" (UniqueName: \"kubernetes.io/projected/7e64cb6c-a574-4272-acc4-5607085c3a2f-kube-api-access-5nxz8\") pod \"ingress-canary-9mdhg\" (UID: \"7e64cb6c-a574-4272-acc4-5607085c3a2f\") " pod="openshift-ingress-canary/ingress-canary-9mdhg"
May 28 17:17:50.844879 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.842645    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert\") pod \"ingress-canary-9mdhg\" (UID: \"7e64cb6c-a574-4272-acc4-5607085c3a2f\") " pod="openshift-ingress-canary/ingress-canary-9mdhg"
May 28 17:17:50.844879 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.842707    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwzjc\" (UniqueName: \"kubernetes.io/projected/bac47327-8e2e-410d-9a4b-53d26f170d1b-kube-api-access-jwzjc\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:17:50.844879 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.842790    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac47327-8e2e-410d-9a4b-53d26f170d1b-tmp-dir\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:17:50.944189 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.944104    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac47327-8e2e-410d-9a4b-53d26f170d1b-tmp-dir\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:17:50.944189 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.944149    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:17:50.944189 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.944186    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bac47327-8e2e-410d-9a4b-53d26f170d1b-config-volume\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:17:50.944776 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.944220    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5nxz8\" (UniqueName: \"kubernetes.io/projected/7e64cb6c-a574-4272-acc4-5607085c3a2f-kube-api-access-5nxz8\") pod \"ingress-canary-9mdhg\" (UID: \"7e64cb6c-a574-4272-acc4-5607085c3a2f\") " pod="openshift-ingress-canary/ingress-canary-9mdhg"
May 28 17:17:50.944776 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.944245    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert\") pod \"ingress-canary-9mdhg\" (UID: \"7e64cb6c-a574-4272-acc4-5607085c3a2f\") " pod="openshift-ingress-canary/ingress-canary-9mdhg"
May 28 17:17:50.944776 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.944261    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jwzjc\" (UniqueName: \"kubernetes.io/projected/bac47327-8e2e-410d-9a4b-53d26f170d1b-kube-api-access-jwzjc\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:17:50.944776 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:50.944296    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:17:50.944776 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:50.944372    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls podName:bac47327-8e2e-410d-9a4b-53d26f170d1b nodeName:}" failed. No retries permitted until 2026-05-28 17:17:51.444350589 +0000 UTC m=+35.683900959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls") pod "dns-default-jn46x" (UID: "bac47327-8e2e-410d-9a4b-53d26f170d1b") : secret "dns-default-metrics-tls" not found
May 28 17:17:50.944776 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.944551    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac47327-8e2e-410d-9a4b-53d26f170d1b-tmp-dir\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:17:50.944776 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:50.944611    2561 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:17:50.944776 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:50.944648    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert podName:7e64cb6c-a574-4272-acc4-5607085c3a2f nodeName:}" failed. No retries permitted until 2026-05-28 17:17:51.444635139 +0000 UTC m=+35.684185507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert") pod "ingress-canary-9mdhg" (UID: "7e64cb6c-a574-4272-acc4-5607085c3a2f") : secret "canary-serving-cert" not found
May 28 17:17:50.945191 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.945133    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bac47327-8e2e-410d-9a4b-53d26f170d1b-config-volume\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:17:50.956170 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.956146    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwzjc\" (UniqueName: \"kubernetes.io/projected/bac47327-8e2e-410d-9a4b-53d26f170d1b-kube-api-access-jwzjc\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:17:50.956334 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:50.956312    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nxz8\" (UniqueName: \"kubernetes.io/projected/7e64cb6c-a574-4272-acc4-5607085c3a2f-kube-api-access-5nxz8\") pod \"ingress-canary-9mdhg\" (UID: \"7e64cb6c-a574-4272-acc4-5607085c3a2f\") " pod="openshift-ingress-canary/ingress-canary-9mdhg"
May 28 17:17:51.447577 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:51.447538    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert\") pod \"ingress-canary-9mdhg\" (UID: \"7e64cb6c-a574-4272-acc4-5607085c3a2f\") " pod="openshift-ingress-canary/ingress-canary-9mdhg"
May 28 17:17:51.447801 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:51.447609    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:17:51.447801 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:51.447724    2561 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:17:51.447801 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:51.447737    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:17:51.447983 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:51.447802    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert podName:7e64cb6c-a574-4272-acc4-5607085c3a2f nodeName:}" failed. No retries permitted until 2026-05-28 17:17:52.447780974 +0000 UTC m=+36.687331341 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert") pod "ingress-canary-9mdhg" (UID: "7e64cb6c-a574-4272-acc4-5607085c3a2f") : secret "canary-serving-cert" not found
May 28 17:17:51.447983 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:51.447823    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls podName:bac47327-8e2e-410d-9a4b-53d26f170d1b nodeName:}" failed. No retries permitted until 2026-05-28 17:17:52.447815284 +0000 UTC m=+36.687365651 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls") pod "dns-default-jn46x" (UID: "bac47327-8e2e-410d-9a4b-53d26f170d1b") : secret "dns-default-metrics-tls" not found
May 28 17:17:52.316784 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:52.316745    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:17:52.316784 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:52.316771    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:17:52.321960 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:52.321387    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 17:17:52.321960 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:52.321404    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 17:17:52.321960 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:52.321454    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rjfcr\""
May 28 17:17:52.321960 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:52.321403    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-svsk4\""
May 28 17:17:52.321960 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:52.321688    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 17:17:52.455651 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:52.455610    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert\") pod \"ingress-canary-9mdhg\" (UID: \"7e64cb6c-a574-4272-acc4-5607085c3a2f\") " pod="openshift-ingress-canary/ingress-canary-9mdhg"
May 28 17:17:52.455852 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:52.455757    2561 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:17:52.455852 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:52.455796    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:17:52.455852 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:52.455819    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert podName:7e64cb6c-a574-4272-acc4-5607085c3a2f nodeName:}" failed. No retries permitted until 2026-05-28 17:17:54.455800537 +0000 UTC m=+38.695350920 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert") pod "ingress-canary-9mdhg" (UID: "7e64cb6c-a574-4272-acc4-5607085c3a2f") : secret "canary-serving-cert" not found
May 28 17:17:52.456015 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:52.455875    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:17:52.456015 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:52.455925    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls podName:bac47327-8e2e-410d-9a4b-53d26f170d1b nodeName:}" failed. No retries permitted until 2026-05-28 17:17:54.455909613 +0000 UTC m=+38.695460004 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls") pod "dns-default-jn46x" (UID: "bac47327-8e2e-410d-9a4b-53d26f170d1b") : secret "dns-default-metrics-tls" not found
May 28 17:17:54.234746 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.234699    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8"]
May 28 17:17:54.252389 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.252358    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"]
May 28 17:17:54.252560 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.252540    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8"
May 28 17:17:54.255293 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.255268    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
May 28 17:17:54.255293 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.255289    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
May 28 17:17:54.255494 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.255308    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
May 28 17:17:54.255494 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.255441    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
May 28 17:17:54.263413 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.263389    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8"]
May 28 17:17:54.263525 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.263420    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"]
May 28 17:17:54.263525 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.263516    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.266699 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.266659    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
May 28 17:17:54.266827 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.266721    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
May 28 17:17:54.266827 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.266660    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
May 28 17:17:54.266939 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.266920    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
May 28 17:17:54.369266 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.369222    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b83fd91b-1faf-4b17-8bce-2fec27fbae5c-tmp\") pod \"klusterlet-addon-workmgr-756b8b84fd-mmft8\" (UID: \"b83fd91b-1faf-4b17-8bce-2fec27fbae5c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8"
May 28 17:17:54.369435 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.369271    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pktvh\" (UniqueName: \"kubernetes.io/projected/b1fe7206-e183-43be-9c22-27dd1066df34-kube-api-access-pktvh\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.369435 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.369385    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b83fd91b-1faf-4b17-8bce-2fec27fbae5c-klusterlet-config\") pod \"klusterlet-addon-workmgr-756b8b84fd-mmft8\" (UID: \"b83fd91b-1faf-4b17-8bce-2fec27fbae5c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8"
May 28 17:17:54.369525 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.369446    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljvh8\" (UniqueName: \"kubernetes.io/projected/b83fd91b-1faf-4b17-8bce-2fec27fbae5c-kube-api-access-ljvh8\") pod \"klusterlet-addon-workmgr-756b8b84fd-mmft8\" (UID: \"b83fd91b-1faf-4b17-8bce-2fec27fbae5c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8"
May 28 17:17:54.369525 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.369473    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1fe7206-e183-43be-9c22-27dd1066df34-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.369525 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.369494    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/b1fe7206-e183-43be-9c22-27dd1066df34-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.369525 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.369518    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b1fe7206-e183-43be-9c22-27dd1066df34-hub\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.369668 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.369535    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b1fe7206-e183-43be-9c22-27dd1066df34-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.369668 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.369588    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b1fe7206-e183-43be-9c22-27dd1066df34-ca\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.465439 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.465341    2561 generic.go:358] "Generic (PLEG): container finished" podID="6ae61aa8-393c-4a9a-a9ae-f125c69e5871" containerID="ac6761513fc57dc4876272eaa93a47e4b0079b08fbbc2dc14edfbafaa9fe6abf" exitCode=0
May 28 17:17:54.465439 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.465401    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l55xd" event={"ID":"6ae61aa8-393c-4a9a-a9ae-f125c69e5871","Type":"ContainerDied","Data":"ac6761513fc57dc4876272eaa93a47e4b0079b08fbbc2dc14edfbafaa9fe6abf"}
May 28 17:17:54.469868 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.469840    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b83fd91b-1faf-4b17-8bce-2fec27fbae5c-klusterlet-config\") pod \"klusterlet-addon-workmgr-756b8b84fd-mmft8\" (UID: \"b83fd91b-1faf-4b17-8bce-2fec27fbae5c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8"
May 28 17:17:54.469985 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.469887    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ljvh8\" (UniqueName: \"kubernetes.io/projected/b83fd91b-1faf-4b17-8bce-2fec27fbae5c-kube-api-access-ljvh8\") pod \"klusterlet-addon-workmgr-756b8b84fd-mmft8\" (UID: \"b83fd91b-1faf-4b17-8bce-2fec27fbae5c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8"
May 28 17:17:54.469985 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.469906    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1fe7206-e183-43be-9c22-27dd1066df34-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.469985 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.469926    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/b1fe7206-e183-43be-9c22-27dd1066df34-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.469985 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.469945    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:17:54.469985 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.469960    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b1fe7206-e183-43be-9c22-27dd1066df34-hub\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.469985 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.469978    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b1fe7206-e183-43be-9c22-27dd1066df34-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.470283 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.470020    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b1fe7206-e183-43be-9c22-27dd1066df34-ca\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.470283 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.470051    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b83fd91b-1faf-4b17-8bce-2fec27fbae5c-tmp\") pod \"klusterlet-addon-workmgr-756b8b84fd-mmft8\" (UID: \"b83fd91b-1faf-4b17-8bce-2fec27fbae5c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8"
May 28 17:17:54.470283 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.470081    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pktvh\" (UniqueName: \"kubernetes.io/projected/b1fe7206-e183-43be-9c22-27dd1066df34-kube-api-access-pktvh\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.470283 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:54.470091    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:17:54.470283 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.470120    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert\") pod \"ingress-canary-9mdhg\" (UID: \"7e64cb6c-a574-4272-acc4-5607085c3a2f\") " pod="openshift-ingress-canary/ingress-canary-9mdhg"
May 28 17:17:54.470283 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:54.470170    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls podName:bac47327-8e2e-410d-9a4b-53d26f170d1b nodeName:}" failed. No retries permitted until 2026-05-28 17:17:58.470147454 +0000 UTC m=+42.709697839 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls") pod "dns-default-jn46x" (UID: "bac47327-8e2e-410d-9a4b-53d26f170d1b") : secret "dns-default-metrics-tls" not found
May 28 17:17:54.470283 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:54.470207    2561 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:17:54.470283 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:54.470263    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert podName:7e64cb6c-a574-4272-acc4-5607085c3a2f nodeName:}" failed. No retries permitted until 2026-05-28 17:17:58.470247425 +0000 UTC m=+42.709797794 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert") pod "ingress-canary-9mdhg" (UID: "7e64cb6c-a574-4272-acc4-5607085c3a2f") : secret "canary-serving-cert" not found
May 28 17:17:54.470656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.470576    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b83fd91b-1faf-4b17-8bce-2fec27fbae5c-tmp\") pod \"klusterlet-addon-workmgr-756b8b84fd-mmft8\" (UID: \"b83fd91b-1faf-4b17-8bce-2fec27fbae5c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8"
May 28 17:17:54.470656 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.470618    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/b1fe7206-e183-43be-9c22-27dd1066df34-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.473388 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.473341    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b1fe7206-e183-43be-9c22-27dd1066df34-ca\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.473595 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.473473    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b1fe7206-e183-43be-9c22-27dd1066df34-hub\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.473595 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.473490    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b83fd91b-1faf-4b17-8bce-2fec27fbae5c-klusterlet-config\") pod \"klusterlet-addon-workmgr-756b8b84fd-mmft8\" (UID: \"b83fd91b-1faf-4b17-8bce-2fec27fbae5c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8"
May 28 17:17:54.473595 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.473574    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b1fe7206-e183-43be-9c22-27dd1066df34-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.473860 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.473841    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1fe7206-e183-43be-9c22-27dd1066df34-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.478734 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.478709    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljvh8\" (UniqueName: \"kubernetes.io/projected/b83fd91b-1faf-4b17-8bce-2fec27fbae5c-kube-api-access-ljvh8\") pod \"klusterlet-addon-workmgr-756b8b84fd-mmft8\" (UID: \"b83fd91b-1faf-4b17-8bce-2fec27fbae5c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8"
May 28 17:17:54.479108 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.479064    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pktvh\" (UniqueName: \"kubernetes.io/projected/b1fe7206-e183-43be-9c22-27dd1066df34-kube-api-access-pktvh\") pod \"cluster-proxy-proxy-agent-5b56b95fcd-gtrzw\" (UID: \"b1fe7206-e183-43be-9c22-27dd1066df34\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.561273 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.561242    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8"
May 28 17:17:54.589311 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.589210    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:17:54.722293 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.722212    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8"]
May 28 17:17:54.737613 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:54.737579    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"]
May 28 17:17:54.741593 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:54.741563    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb83fd91b_1faf_4b17_8bce_2fec27fbae5c.slice/crio-846707cfc942b191de84a81a61d50bc9ede0b2f401e80538587e7d0ed63e3422 WatchSource:0}: Error finding container 846707cfc942b191de84a81a61d50bc9ede0b2f401e80538587e7d0ed63e3422: Status 404 returned error can't find the container with id 846707cfc942b191de84a81a61d50bc9ede0b2f401e80538587e7d0ed63e3422
May 28 17:17:54.741837 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:17:54.741820    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1fe7206_e183_43be_9c22_27dd1066df34.slice/crio-a3fa3d29ec7fdc6b2170a50ed4235f0749d225bdfbd289ad7dcd10b4924dc042 WatchSource:0}: Error finding container a3fa3d29ec7fdc6b2170a50ed4235f0749d225bdfbd289ad7dcd10b4924dc042: Status 404 returned error can't find the container with id a3fa3d29ec7fdc6b2170a50ed4235f0749d225bdfbd289ad7dcd10b4924dc042
May 28 17:17:55.473513 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:55.473473    2561 generic.go:358] "Generic (PLEG): container finished" podID="6ae61aa8-393c-4a9a-a9ae-f125c69e5871" containerID="0423bf1beb45af24418b30cfc9bb8aee121b9b956b96ae2dd2b13d6c2d3335be" exitCode=0
May 28 17:17:55.474227 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:55.473551    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l55xd" event={"ID":"6ae61aa8-393c-4a9a-a9ae-f125c69e5871","Type":"ContainerDied","Data":"0423bf1beb45af24418b30cfc9bb8aee121b9b956b96ae2dd2b13d6c2d3335be"}
May 28 17:17:55.475223 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:55.475190    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw" event={"ID":"b1fe7206-e183-43be-9c22-27dd1066df34","Type":"ContainerStarted","Data":"a3fa3d29ec7fdc6b2170a50ed4235f0749d225bdfbd289ad7dcd10b4924dc042"}
May 28 17:17:55.476371 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:55.476348    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8" event={"ID":"b83fd91b-1faf-4b17-8bce-2fec27fbae5c","Type":"ContainerStarted","Data":"846707cfc942b191de84a81a61d50bc9ede0b2f401e80538587e7d0ed63e3422"}
May 28 17:17:56.482920 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:56.482878    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l55xd" event={"ID":"6ae61aa8-393c-4a9a-a9ae-f125c69e5871","Type":"ContainerStarted","Data":"8ee5d10b994083e3d75748b5c9422f7874594a9947bed045220ae751950e02e3"}
May 28 17:17:56.509527 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:56.509461    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-l55xd" podStartSLOduration=5.739851116 podStartE2EDuration="40.509441184s" podCreationTimestamp="2026-05-28 17:17:16 +0000 UTC" firstStartedPulling="2026-05-28 17:17:19.165268024 +0000 UTC m=+3.404818392" lastFinishedPulling="2026-05-28 17:17:53.934858092 +0000 UTC m=+38.174408460" observedRunningTime="2026-05-28 17:17:56.507692783 +0000 UTC m=+40.747243168" watchObservedRunningTime="2026-05-28 17:17:56.509441184 +0000 UTC m=+40.748991575"
May 28 17:17:58.505242 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:58.505206    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert\") pod \"ingress-canary-9mdhg\" (UID: \"7e64cb6c-a574-4272-acc4-5607085c3a2f\") " pod="openshift-ingress-canary/ingress-canary-9mdhg"
May 28 17:17:58.505724 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:17:58.505297    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:17:58.505724 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:58.505377    2561 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:17:58.505724 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:58.505410    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:17:58.505724 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:58.505454    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert podName:7e64cb6c-a574-4272-acc4-5607085c3a2f nodeName:}" failed. No retries permitted until 2026-05-28 17:18:06.5054349 +0000 UTC m=+50.744985285 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert") pod "ingress-canary-9mdhg" (UID: "7e64cb6c-a574-4272-acc4-5607085c3a2f") : secret "canary-serving-cert" not found
May 28 17:17:58.505724 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:17:58.505475    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls podName:bac47327-8e2e-410d-9a4b-53d26f170d1b nodeName:}" failed. No retries permitted until 2026-05-28 17:18:06.505465676 +0000 UTC m=+50.745016051 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls") pod "dns-default-jn46x" (UID: "bac47327-8e2e-410d-9a4b-53d26f170d1b") : secret "dns-default-metrics-tls" not found
May 28 17:18:00.491874 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:00.491833    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8" event={"ID":"b83fd91b-1faf-4b17-8bce-2fec27fbae5c","Type":"ContainerStarted","Data":"c46d3c55251287700410a995a10149c3cd7c891a9402f3a465ea9477e87b27e4"}
May 28 17:18:00.492404 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:00.492015    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8"
May 28 17:18:00.493146 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:00.493119    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw" event={"ID":"b1fe7206-e183-43be-9c22-27dd1066df34","Type":"ContainerStarted","Data":"50b1f5e7660c27e81284d7850025b1e1089b25b3d35b8f149dc2f469abe7ef0c"}
May 28 17:18:00.493921 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:00.493904    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8"
May 28 17:18:00.528416 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:00.528373    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-756b8b84fd-mmft8" podStartSLOduration=1.453478954 podStartE2EDuration="6.528360302s" podCreationTimestamp="2026-05-28 17:17:54 +0000 UTC" firstStartedPulling="2026-05-28 17:17:54.743106616 +0000 UTC m=+38.982656984" lastFinishedPulling="2026-05-28 17:17:59.817987964 +0000 UTC m=+44.057538332" observedRunningTime="2026-05-28 17:18:00.510526968 +0000 UTC m=+44.750077359" watchObservedRunningTime="2026-05-28 17:18:00.528360302 +0000 UTC m=+44.767910693"
May 28 17:18:02.499082 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:02.499043    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw" event={"ID":"b1fe7206-e183-43be-9c22-27dd1066df34","Type":"ContainerStarted","Data":"4a296dcbfdd408da1726e5be77570c437cfbd41b0395fe8cb6156c442221b875"}
May 28 17:18:02.499082 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:02.499080    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw" event={"ID":"b1fe7206-e183-43be-9c22-27dd1066df34","Type":"ContainerStarted","Data":"8d05420e94da4c7d4196a7077e230006824921c4c29d868207071593253fb195"}
May 28 17:18:02.521614 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:02.521528    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw" podStartSLOduration=1.014384468 podStartE2EDuration="8.521514323s" podCreationTimestamp="2026-05-28 17:17:54 +0000 UTC" firstStartedPulling="2026-05-28 17:17:54.743557874 +0000 UTC m=+38.983108243" lastFinishedPulling="2026-05-28 17:18:02.250687716 +0000 UTC m=+46.490238098" observedRunningTime="2026-05-28 17:18:02.519700656 +0000 UTC m=+46.759251043" watchObservedRunningTime="2026-05-28 17:18:02.521514323 +0000 UTC m=+46.761064712"
May 28 17:18:06.564742 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:06.564661    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert\") pod \"ingress-canary-9mdhg\" (UID: \"7e64cb6c-a574-4272-acc4-5607085c3a2f\") " pod="openshift-ingress-canary/ingress-canary-9mdhg"
May 28 17:18:06.565194 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:06.564787    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:18:06.565194 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:18:06.564824    2561 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:18:06.565194 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:18:06.564895    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:18:06.565194 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:18:06.564896    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert podName:7e64cb6c-a574-4272-acc4-5607085c3a2f nodeName:}" failed. No retries permitted until 2026-05-28 17:18:22.564877246 +0000 UTC m=+66.804427616 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert") pod "ingress-canary-9mdhg" (UID: "7e64cb6c-a574-4272-acc4-5607085c3a2f") : secret "canary-serving-cert" not found
May 28 17:18:06.565194 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:18:06.564935    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls podName:bac47327-8e2e-410d-9a4b-53d26f170d1b nodeName:}" failed. No retries permitted until 2026-05-28 17:18:22.564925609 +0000 UTC m=+66.804475983 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls") pod "dns-default-jn46x" (UID: "bac47327-8e2e-410d-9a4b-53d26f170d1b") : secret "dns-default-metrics-tls" not found
May 28 17:18:15.467213 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:15.467178    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5gxv8"
May 28 17:18:21.981760 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:21.981719    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs\") pod \"network-metrics-daemon-7xrml\" (UID: \"1a8e2081-5b89-4115-82b2-d1afbf6f58ed\") " pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:18:21.985028 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:21.984998    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 17:18:21.992885 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:18:21.992862    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 17:18:21.992992 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:18:21.992922    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs podName:1a8e2081-5b89-4115-82b2-d1afbf6f58ed nodeName:}" failed. No retries permitted until 2026-05-28 17:19:25.992906492 +0000 UTC m=+130.232456861 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs") pod "network-metrics-daemon-7xrml" (UID: "1a8e2081-5b89-4115-82b2-d1afbf6f58ed") : secret "metrics-daemon-secret" not found
May 28 17:18:22.082848 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:22.082813    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qxnhb\" (UniqueName: \"kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb\") pod \"network-check-target-hrcwh\" (UID: \"b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4\") " pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:18:22.086377 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:22.086354    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 17:18:22.096348 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:22.096324    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 17:18:22.106863 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:22.106841    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxnhb\" (UniqueName: \"kubernetes.io/projected/b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4-kube-api-access-qxnhb\") pod \"network-check-target-hrcwh\" (UID: \"b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4\") " pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:18:22.332052 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:22.332025    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rjfcr\""
May 28 17:18:22.339523 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:22.339508    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:18:22.459558 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:22.459527    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hrcwh"]
May 28 17:18:22.462584 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:18:22.462546    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4ba2aa4_a73a_402c_8a26_fa03ad55a9b4.slice/crio-42baef9126bed562f57dc6655a6f8ea89ccda3cdb47a472a884f67358057c606 WatchSource:0}: Error finding container 42baef9126bed562f57dc6655a6f8ea89ccda3cdb47a472a884f67358057c606: Status 404 returned error can't find the container with id 42baef9126bed562f57dc6655a6f8ea89ccda3cdb47a472a884f67358057c606
May 28 17:18:22.537455 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:22.537415    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hrcwh" event={"ID":"b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4","Type":"ContainerStarted","Data":"42baef9126bed562f57dc6655a6f8ea89ccda3cdb47a472a884f67358057c606"}
May 28 17:18:22.586890 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:22.586814    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert\") pod \"ingress-canary-9mdhg\" (UID: \"7e64cb6c-a574-4272-acc4-5607085c3a2f\") " pod="openshift-ingress-canary/ingress-canary-9mdhg"
May 28 17:18:22.586890 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:22.586876    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:18:22.587018 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:18:22.586958    2561 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:18:22.587018 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:18:22.586971    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:18:22.587094 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:18:22.587020    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert podName:7e64cb6c-a574-4272-acc4-5607085c3a2f nodeName:}" failed. No retries permitted until 2026-05-28 17:18:54.58700485 +0000 UTC m=+98.826555218 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert") pod "ingress-canary-9mdhg" (UID: "7e64cb6c-a574-4272-acc4-5607085c3a2f") : secret "canary-serving-cert" not found
May 28 17:18:22.587094 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:18:22.587036    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls podName:bac47327-8e2e-410d-9a4b-53d26f170d1b nodeName:}" failed. No retries permitted until 2026-05-28 17:18:54.587029807 +0000 UTC m=+98.826580174 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls") pod "dns-default-jn46x" (UID: "bac47327-8e2e-410d-9a4b-53d26f170d1b") : secret "dns-default-metrics-tls" not found
May 28 17:18:25.547158 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:25.547067    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hrcwh" event={"ID":"b4ba2aa4-a73a-402c-8a26-fa03ad55a9b4","Type":"ContainerStarted","Data":"a4be4fe63f06d01042ae3925800dfda798b660846ef1e9a4067e4eb7f98c8028"}
May 28 17:18:25.547493 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:25.547216    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:18:25.567144 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:25.567095    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hrcwh" podStartSLOduration=66.824292714 podStartE2EDuration="1m9.567081015s" podCreationTimestamp="2026-05-28 17:17:16 +0000 UTC" firstStartedPulling="2026-05-28 17:18:22.464541893 +0000 UTC m=+66.704092260" lastFinishedPulling="2026-05-28 17:18:25.207330193 +0000 UTC m=+69.446880561" observedRunningTime="2026-05-28 17:18:25.565624912 +0000 UTC m=+69.805175304" watchObservedRunningTime="2026-05-28 17:18:25.567081015 +0000 UTC m=+69.806631404"
May 28 17:18:32.686241 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:32.686121    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vc2ht_4222bafb-9e5f-4884-832d-83364e16f3e9/dns-node-resolver/0.log"
May 28 17:18:33.285223 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:33.285193    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kmg5m_cf00e197-64d4-4961-9730-248b6e1ec010/node-ca/0.log"
May 28 17:18:54.621397 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:54.621352    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:18:54.621787 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:54.621411    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert\") pod \"ingress-canary-9mdhg\" (UID: \"7e64cb6c-a574-4272-acc4-5607085c3a2f\") " pod="openshift-ingress-canary/ingress-canary-9mdhg"
May 28 17:18:54.623846 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:54.623821    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bac47327-8e2e-410d-9a4b-53d26f170d1b-metrics-tls\") pod \"dns-default-jn46x\" (UID: \"bac47327-8e2e-410d-9a4b-53d26f170d1b\") " pod="openshift-dns/dns-default-jn46x"
May 28 17:18:54.623949 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:54.623860    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e64cb6c-a574-4272-acc4-5607085c3a2f-cert\") pod \"ingress-canary-9mdhg\" (UID: \"7e64cb6c-a574-4272-acc4-5607085c3a2f\") " pod="openshift-ingress-canary/ingress-canary-9mdhg"
May 28 17:18:54.635464 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:54.635442    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sh6sk\""
May 28 17:18:54.642842 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:54.642829    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jn46x"
May 28 17:18:54.650179 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:54.650161    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nznc5\""
May 28 17:18:54.657726 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:54.657702    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9mdhg"
May 28 17:18:54.776243 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:54.776213    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jn46x"]
May 28 17:18:54.779584 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:18:54.779554    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbac47327_8e2e_410d_9a4b_53d26f170d1b.slice/crio-f84178499db09505c12208bd8ac124f1d484d38bd2eeac558ecfe7e74b1389f1 WatchSource:0}: Error finding container f84178499db09505c12208bd8ac124f1d484d38bd2eeac558ecfe7e74b1389f1: Status 404 returned error can't find the container with id f84178499db09505c12208bd8ac124f1d484d38bd2eeac558ecfe7e74b1389f1
May 28 17:18:54.790568 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:54.790548    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9mdhg"]
May 28 17:18:54.793732 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:18:54.793707    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e64cb6c_a574_4272_acc4_5607085c3a2f.slice/crio-d95f6a76018ebc60eba3f488e1051384086bcdf808cd5f8135bc0ba8374e7eea WatchSource:0}: Error finding container d95f6a76018ebc60eba3f488e1051384086bcdf808cd5f8135bc0ba8374e7eea: Status 404 returned error can't find the container with id d95f6a76018ebc60eba3f488e1051384086bcdf808cd5f8135bc0ba8374e7eea
May 28 17:18:55.605338 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:55.605265    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9mdhg" event={"ID":"7e64cb6c-a574-4272-acc4-5607085c3a2f","Type":"ContainerStarted","Data":"d95f6a76018ebc60eba3f488e1051384086bcdf808cd5f8135bc0ba8374e7eea"}
May 28 17:18:55.606693 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:55.606645    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jn46x" event={"ID":"bac47327-8e2e-410d-9a4b-53d26f170d1b","Type":"ContainerStarted","Data":"f84178499db09505c12208bd8ac124f1d484d38bd2eeac558ecfe7e74b1389f1"}
May 28 17:18:56.551895 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:56.551866    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hrcwh"
May 28 17:18:57.612427 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:57.612389    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9mdhg" event={"ID":"7e64cb6c-a574-4272-acc4-5607085c3a2f","Type":"ContainerStarted","Data":"4d4690460755f084ab752470930683ac72df18136fc75c26804062d74dcf30e9"}
May 28 17:18:57.613988 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:57.613962    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jn46x" event={"ID":"bac47327-8e2e-410d-9a4b-53d26f170d1b","Type":"ContainerStarted","Data":"6dd856a7f88cc39407eaf99bedd212a82cc3954e0bb6c740d2eb89063e9fa55d"}
May 28 17:18:57.614121 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:57.613992    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jn46x" event={"ID":"bac47327-8e2e-410d-9a4b-53d26f170d1b","Type":"ContainerStarted","Data":"5410bd77c8bf7e513fd520d4f4a90faedc67417ec3edb93a2441441390773bbf"}
May 28 17:18:57.614121 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:57.614062    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jn46x"
May 28 17:18:57.630657 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:57.630593    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9mdhg" podStartSLOduration=65.747962259 podStartE2EDuration="1m7.630580655s" podCreationTimestamp="2026-05-28 17:17:50 +0000 UTC" firstStartedPulling="2026-05-28 17:18:54.795368049 +0000 UTC m=+99.034918419" lastFinishedPulling="2026-05-28 17:18:56.677986443 +0000 UTC m=+100.917536815" observedRunningTime="2026-05-28 17:18:57.630077846 +0000 UTC m=+101.869628239" watchObservedRunningTime="2026-05-28 17:18:57.630580655 +0000 UTC m=+101.870131045"
May 28 17:18:57.649038 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:18:57.648987    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-jn46x" podStartSLOduration=65.754280457 podStartE2EDuration="1m7.648971206s" podCreationTimestamp="2026-05-28 17:17:50 +0000 UTC" firstStartedPulling="2026-05-28 17:18:54.781273443 +0000 UTC m=+99.020823811" lastFinishedPulling="2026-05-28 17:18:56.675964193 +0000 UTC m=+100.915514560" observedRunningTime="2026-05-28 17:18:57.648286927 +0000 UTC m=+101.887837327" watchObservedRunningTime="2026-05-28 17:18:57.648971206 +0000 UTC m=+101.888521595"
May 28 17:19:05.036846 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.036806    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7467f99487-r96cs"]
May 28 17:19:05.039693 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.039657    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.044772 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.044736    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 28 17:19:05.044907 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.044801    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xv2ct\""
May 28 17:19:05.045430 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.045410    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 28 17:19:05.045510 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.045456    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 28 17:19:05.052233 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.052210    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zbkcs"]
May 28 17:19:05.052778 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.052759    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 28 17:19:05.054888 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.054874    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.057837 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.057814    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 28 17:19:05.057937 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.057818    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9mq7s\""
May 28 17:19:05.058034 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.058016    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 28 17:19:05.058117 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.058101    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 28 17:19:05.058176 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.058155    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 28 17:19:05.063536 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.063517    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7467f99487-r96cs"]
May 28 17:19:05.077647 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.077616    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zbkcs"]
May 28 17:19:05.192555 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.192515    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/440788b2-ab2b-413e-8150-482e0cb1b920-ca-trust-extracted\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.192555 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.192556    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h9n4\" (UniqueName: \"kubernetes.io/projected/440788b2-ab2b-413e-8150-482e0cb1b920-kube-api-access-9h9n4\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.192808 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.192589    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3-crio-socket\") pod \"insights-runtime-extractor-zbkcs\" (UID: \"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3\") " pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.192808 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.192640    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/440788b2-ab2b-413e-8150-482e0cb1b920-trusted-ca\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.192808 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.192695    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3-data-volume\") pod \"insights-runtime-extractor-zbkcs\" (UID: \"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3\") " pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.192808 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.192715    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nd74\" (UniqueName: \"kubernetes.io/projected/3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3-kube-api-access-7nd74\") pod \"insights-runtime-extractor-zbkcs\" (UID: \"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3\") " pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.192808 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.192781    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/440788b2-ab2b-413e-8150-482e0cb1b920-image-registry-private-configuration\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.192987 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.192822    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/440788b2-ab2b-413e-8150-482e0cb1b920-registry-tls\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.192987 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.192838    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/440788b2-ab2b-413e-8150-482e0cb1b920-bound-sa-token\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.192987 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.192854    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zbkcs\" (UID: \"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3\") " pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.192987 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.192872    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/440788b2-ab2b-413e-8150-482e0cb1b920-installation-pull-secrets\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.192987 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.192887    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zbkcs\" (UID: \"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3\") " pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.192987 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.192915    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/440788b2-ab2b-413e-8150-482e0cb1b920-registry-certificates\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.293723 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.293603    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/440788b2-ab2b-413e-8150-482e0cb1b920-ca-trust-extracted\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.293723 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.293647    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9n4\" (UniqueName: \"kubernetes.io/projected/440788b2-ab2b-413e-8150-482e0cb1b920-kube-api-access-9h9n4\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.293957 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.293783    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3-crio-socket\") pod \"insights-runtime-extractor-zbkcs\" (UID: \"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3\") " pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.293957 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.293815    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/440788b2-ab2b-413e-8150-482e0cb1b920-trusted-ca\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.293957 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.293830    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3-data-volume\") pod \"insights-runtime-extractor-zbkcs\" (UID: \"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3\") " pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.293957 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.293848    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7nd74\" (UniqueName: \"kubernetes.io/projected/3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3-kube-api-access-7nd74\") pod \"insights-runtime-extractor-zbkcs\" (UID: \"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3\") " pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.293957 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.293889    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/440788b2-ab2b-413e-8150-482e0cb1b920-image-registry-private-configuration\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.293957 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.293899    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3-crio-socket\") pod \"insights-runtime-extractor-zbkcs\" (UID: \"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3\") " pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.293957 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.293937    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/440788b2-ab2b-413e-8150-482e0cb1b920-registry-tls\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.294277 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.293961    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/440788b2-ab2b-413e-8150-482e0cb1b920-bound-sa-token\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.294277 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.293987    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zbkcs\" (UID: \"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3\") " pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.294277 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.294017    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/440788b2-ab2b-413e-8150-482e0cb1b920-installation-pull-secrets\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.294277 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.294040    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zbkcs\" (UID: \"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3\") " pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.294277 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.294080    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/440788b2-ab2b-413e-8150-482e0cb1b920-registry-certificates\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.294277 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.294099    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/440788b2-ab2b-413e-8150-482e0cb1b920-ca-trust-extracted\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.294277 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.294226    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3-data-volume\") pod \"insights-runtime-extractor-zbkcs\" (UID: \"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3\") " pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.294731 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.294706    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zbkcs\" (UID: \"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3\") " pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.294985 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.294919    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/440788b2-ab2b-413e-8150-482e0cb1b920-registry-certificates\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.295184 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.295166    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/440788b2-ab2b-413e-8150-482e0cb1b920-trusted-ca\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.296771 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.296749    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zbkcs\" (UID: \"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3\") " pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.297335 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.296898    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/440788b2-ab2b-413e-8150-482e0cb1b920-installation-pull-secrets\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.297335 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.297140    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/440788b2-ab2b-413e-8150-482e0cb1b920-image-registry-private-configuration\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.297335 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.297216    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/440788b2-ab2b-413e-8150-482e0cb1b920-registry-tls\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.302300 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.302279    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/440788b2-ab2b-413e-8150-482e0cb1b920-bound-sa-token\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.302815 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.302790    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nd74\" (UniqueName: \"kubernetes.io/projected/3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3-kube-api-access-7nd74\") pod \"insights-runtime-extractor-zbkcs\" (UID: \"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3\") " pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.304536 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.304499    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h9n4\" (UniqueName: \"kubernetes.io/projected/440788b2-ab2b-413e-8150-482e0cb1b920-kube-api-access-9h9n4\") pod \"image-registry-7467f99487-r96cs\" (UID: \"440788b2-ab2b-413e-8150-482e0cb1b920\") " pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.348836 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.348791    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.363574 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.363546    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zbkcs"
May 28 17:19:05.500400 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.500377    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7467f99487-r96cs"]
May 28 17:19:05.502874 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:19:05.502846    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod440788b2_ab2b_413e_8150_482e0cb1b920.slice/crio-0833e8b50d6f64d217d527eb76cec09231a97e2017cead2d314a955e69536b4e WatchSource:0}: Error finding container 0833e8b50d6f64d217d527eb76cec09231a97e2017cead2d314a955e69536b4e: Status 404 returned error can't find the container with id 0833e8b50d6f64d217d527eb76cec09231a97e2017cead2d314a955e69536b4e
May 28 17:19:05.516894 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.516874    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zbkcs"]
May 28 17:19:05.520493 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:19:05.520469    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ac46bd6_0a04_4f80_bc16_d38a2e8b76f3.slice/crio-320a1e18fa530a726ed87f165b51a6f81295d49c96720454aafe04ddc7f01650 WatchSource:0}: Error finding container 320a1e18fa530a726ed87f165b51a6f81295d49c96720454aafe04ddc7f01650: Status 404 returned error can't find the container with id 320a1e18fa530a726ed87f165b51a6f81295d49c96720454aafe04ddc7f01650
May 28 17:19:05.633123 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.633080    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zbkcs" event={"ID":"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3","Type":"ContainerStarted","Data":"b9a805095d9c80f7e59a68f044176ec49daaa9e9c8a438295100e5c7030d375a"}
May 28 17:19:05.633123 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.633125    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zbkcs" event={"ID":"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3","Type":"ContainerStarted","Data":"320a1e18fa530a726ed87f165b51a6f81295d49c96720454aafe04ddc7f01650"}
May 28 17:19:05.634399 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.634359    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7467f99487-r96cs" event={"ID":"440788b2-ab2b-413e-8150-482e0cb1b920","Type":"ContainerStarted","Data":"73c9f463da323e532c1c1d6e49413394b92f860f20457fe94c147cf4b7773cd1"}
May 28 17:19:05.634399 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.634401    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7467f99487-r96cs" event={"ID":"440788b2-ab2b-413e-8150-482e0cb1b920","Type":"ContainerStarted","Data":"0833e8b50d6f64d217d527eb76cec09231a97e2017cead2d314a955e69536b4e"}
May 28 17:19:05.634583 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.634506    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:05.657317 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:05.657273    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7467f99487-r96cs" podStartSLOduration=0.657259253 podStartE2EDuration="657.259253ms" podCreationTimestamp="2026-05-28 17:19:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:19:05.656561199 +0000 UTC m=+109.896111613" watchObservedRunningTime="2026-05-28 17:19:05.657259253 +0000 UTC m=+109.896809643"
May 28 17:19:06.639207 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:06.639171    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zbkcs" event={"ID":"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3","Type":"ContainerStarted","Data":"26f91534d94c7b3d713b0e5ce7003be2faa5f2bc4e855e8f34e2cf1c81e0bb9d"}
May 28 17:19:07.618520 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:07.618492    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jn46x"
May 28 17:19:07.642596 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:07.642560    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zbkcs" event={"ID":"3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3","Type":"ContainerStarted","Data":"0cb22a9925b64befe9648cae09e539c368e0a989fcfb91f970ecc163a3d3fd7e"}
May 28 17:19:07.662005 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:07.661964    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zbkcs" podStartSLOduration=0.80418286 podStartE2EDuration="2.661951389s" podCreationTimestamp="2026-05-28 17:19:05 +0000 UTC" firstStartedPulling="2026-05-28 17:19:05.561781637 +0000 UTC m=+109.801332005" lastFinishedPulling="2026-05-28 17:19:07.419550164 +0000 UTC m=+111.659100534" observedRunningTime="2026-05-28 17:19:07.660815058 +0000 UTC m=+111.900365450" watchObservedRunningTime="2026-05-28 17:19:07.661951389 +0000 UTC m=+111.901501778"
May 28 17:19:13.510019 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.509982    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qgz26"]
May 28 17:19:13.512862 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.512756    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.515780 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.515759    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9sdlp\""
May 28 17:19:13.515965 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.515948    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 28 17:19:13.516855 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.516839    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 28 17:19:13.516960 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.516943    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 28 17:19:13.517015 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.516957    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 28 17:19:13.517015 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.516974    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 28 17:19:13.517015 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.516961    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 28 17:19:13.661593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.661561    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/308168d3-91ea-4330-9d54-e47e5d132813-metrics-client-ca\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.661593 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.661595    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/308168d3-91ea-4330-9d54-e47e5d132813-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.661834 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.661612    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwt8g\" (UniqueName: \"kubernetes.io/projected/308168d3-91ea-4330-9d54-e47e5d132813-kube-api-access-kwt8g\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.661834 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.661739    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/308168d3-91ea-4330-9d54-e47e5d132813-node-exporter-textfile\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.661834 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.661768    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/308168d3-91ea-4330-9d54-e47e5d132813-node-exporter-accelerators-collector-config\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.661834 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.661787    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/308168d3-91ea-4330-9d54-e47e5d132813-sys\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.661834 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.661813    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/308168d3-91ea-4330-9d54-e47e5d132813-root\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.662009 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.661868    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/308168d3-91ea-4330-9d54-e47e5d132813-node-exporter-wtmp\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.662009 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.661895    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/308168d3-91ea-4330-9d54-e47e5d132813-node-exporter-tls\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.763118 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.763033    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/308168d3-91ea-4330-9d54-e47e5d132813-node-exporter-wtmp\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.763118 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.763071    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/308168d3-91ea-4330-9d54-e47e5d132813-node-exporter-tls\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.763118 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.763112    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/308168d3-91ea-4330-9d54-e47e5d132813-metrics-client-ca\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.763354 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.763133    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/308168d3-91ea-4330-9d54-e47e5d132813-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.763354 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.763150    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kwt8g\" (UniqueName: \"kubernetes.io/projected/308168d3-91ea-4330-9d54-e47e5d132813-kube-api-access-kwt8g\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.763354 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.763190    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/308168d3-91ea-4330-9d54-e47e5d132813-node-exporter-textfile\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.763354 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.763201    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/308168d3-91ea-4330-9d54-e47e5d132813-node-exporter-wtmp\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.763354 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.763248    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/308168d3-91ea-4330-9d54-e47e5d132813-node-exporter-accelerators-collector-config\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.763354 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.763292    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/308168d3-91ea-4330-9d54-e47e5d132813-sys\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.763354 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.763325    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/308168d3-91ea-4330-9d54-e47e5d132813-root\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.763666 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.763393    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/308168d3-91ea-4330-9d54-e47e5d132813-sys\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.763666 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.763400    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/308168d3-91ea-4330-9d54-e47e5d132813-root\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.763666 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.763556    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/308168d3-91ea-4330-9d54-e47e5d132813-node-exporter-textfile\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.763939 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.763910    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/308168d3-91ea-4330-9d54-e47e5d132813-metrics-client-ca\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.764018 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.764006    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/308168d3-91ea-4330-9d54-e47e5d132813-node-exporter-accelerators-collector-config\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.765691 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.765661    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/308168d3-91ea-4330-9d54-e47e5d132813-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.765808 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.765792    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/308168d3-91ea-4330-9d54-e47e5d132813-node-exporter-tls\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.771937 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.771918    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwt8g\" (UniqueName: \"kubernetes.io/projected/308168d3-91ea-4330-9d54-e47e5d132813-kube-api-access-kwt8g\") pod \"node-exporter-qgz26\" (UID: \"308168d3-91ea-4330-9d54-e47e5d132813\") " pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.822006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:13.821960    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qgz26"
May 28 17:19:13.830193 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:19:13.830154    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308168d3_91ea_4330_9d54_e47e5d132813.slice/crio-15708158aef2589559b7b1f1395ed023b31bc040b5d54b688ad5bbba86965aaa WatchSource:0}: Error finding container 15708158aef2589559b7b1f1395ed023b31bc040b5d54b688ad5bbba86965aaa: Status 404 returned error can't find the container with id 15708158aef2589559b7b1f1395ed023b31bc040b5d54b688ad5bbba86965aaa
May 28 17:19:14.661668 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:14.661631    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qgz26" event={"ID":"308168d3-91ea-4330-9d54-e47e5d132813","Type":"ContainerStarted","Data":"0505d36d6e22f54dfeaef6d98adc6f47dfe893c433dd7fbe83423e67f7abff25"}
May 28 17:19:14.662016 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:14.661694    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qgz26" event={"ID":"308168d3-91ea-4330-9d54-e47e5d132813","Type":"ContainerStarted","Data":"15708158aef2589559b7b1f1395ed023b31bc040b5d54b688ad5bbba86965aaa"}
May 28 17:19:15.666003 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:15.665967    2561 generic.go:358] "Generic (PLEG): container finished" podID="308168d3-91ea-4330-9d54-e47e5d132813" containerID="0505d36d6e22f54dfeaef6d98adc6f47dfe893c433dd7fbe83423e67f7abff25" exitCode=0
May 28 17:19:15.666422 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:15.666040    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qgz26" event={"ID":"308168d3-91ea-4330-9d54-e47e5d132813","Type":"ContainerDied","Data":"0505d36d6e22f54dfeaef6d98adc6f47dfe893c433dd7fbe83423e67f7abff25"}
May 28 17:19:16.670483 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:16.670447    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qgz26" event={"ID":"308168d3-91ea-4330-9d54-e47e5d132813","Type":"ContainerStarted","Data":"c4c2b8853d024f1749a54d93bffd864b09c11daafc9fe015a3525a32ed643244"}
May 28 17:19:16.670883 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:16.670489    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qgz26" event={"ID":"308168d3-91ea-4330-9d54-e47e5d132813","Type":"ContainerStarted","Data":"a7574242aa15d749cde8413eec2e005cc530c2d74729f176db919d486bf57e67"}
May 28 17:19:16.693410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:16.693355    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qgz26" podStartSLOduration=2.950262545 podStartE2EDuration="3.693342293s" podCreationTimestamp="2026-05-28 17:19:13 +0000 UTC" firstStartedPulling="2026-05-28 17:19:13.832287513 +0000 UTC m=+118.071837890" lastFinishedPulling="2026-05-28 17:19:14.575367269 +0000 UTC m=+118.814917638" observedRunningTime="2026-05-28 17:19:16.691569003 +0000 UTC m=+120.931119393" watchObservedRunningTime="2026-05-28 17:19:16.693342293 +0000 UTC m=+120.932892714"
May 28 17:19:17.803994 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.803956    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-57584b7c88-llqzq"]
May 28 17:19:17.806085 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.806069    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.809910 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.809889    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
May 28 17:19:17.810062 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.809958    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-b5mlv\""
May 28 17:19:17.810062 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.809998    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
May 28 17:19:17.810280 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.810265    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
May 28 17:19:17.810375 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.810360    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-9oe7270c1er4b\""
May 28 17:19:17.810416 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.810370    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 28 17:19:17.816081 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.816062    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-57584b7c88-llqzq"]
May 28 17:19:17.894046 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.894015    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/31e62ced-916f-4f53-a789-64d173640f23-secret-metrics-server-client-certs\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.894214 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.894073    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31e62ced-916f-4f53-a789-64d173640f23-client-ca-bundle\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.894214 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.894094    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/31e62ced-916f-4f53-a789-64d173640f23-metrics-server-audit-profiles\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.894214 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.894116    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31e62ced-916f-4f53-a789-64d173640f23-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.894214 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.894147    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29qxw\" (UniqueName: \"kubernetes.io/projected/31e62ced-916f-4f53-a789-64d173640f23-kube-api-access-29qxw\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.894214 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.894168    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/31e62ced-916f-4f53-a789-64d173640f23-secret-metrics-server-tls\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.894214 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.894182    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/31e62ced-916f-4f53-a789-64d173640f23-audit-log\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.995414 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.995380    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-29qxw\" (UniqueName: \"kubernetes.io/projected/31e62ced-916f-4f53-a789-64d173640f23-kube-api-access-29qxw\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.995600 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.995427    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/31e62ced-916f-4f53-a789-64d173640f23-secret-metrics-server-tls\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.995600 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.995540    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/31e62ced-916f-4f53-a789-64d173640f23-audit-log\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.995728 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.995605    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/31e62ced-916f-4f53-a789-64d173640f23-secret-metrics-server-client-certs\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.995728 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.995707    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31e62ced-916f-4f53-a789-64d173640f23-client-ca-bundle\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.995840 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.995747    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/31e62ced-916f-4f53-a789-64d173640f23-metrics-server-audit-profiles\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.995840 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.995777    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31e62ced-916f-4f53-a789-64d173640f23-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.996037 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.996002    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/31e62ced-916f-4f53-a789-64d173640f23-audit-log\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.996475 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.996453    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31e62ced-916f-4f53-a789-64d173640f23-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.996738 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.996622    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/31e62ced-916f-4f53-a789-64d173640f23-metrics-server-audit-profiles\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.998130 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.998110    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/31e62ced-916f-4f53-a789-64d173640f23-secret-metrics-server-tls\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.998820 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.998797    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31e62ced-916f-4f53-a789-64d173640f23-client-ca-bundle\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:17.998901 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:17.998852    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/31e62ced-916f-4f53-a789-64d173640f23-secret-metrics-server-client-certs\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:18.004052 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:18.004032    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29qxw\" (UniqueName: \"kubernetes.io/projected/31e62ced-916f-4f53-a789-64d173640f23-kube-api-access-29qxw\") pod \"metrics-server-57584b7c88-llqzq\" (UID: \"31e62ced-916f-4f53-a789-64d173640f23\") " pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:18.115225 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:18.115191    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:18.239360 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:18.239324    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-57584b7c88-llqzq"]
May 28 17:19:18.243760 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:19:18.243725    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e62ced_916f_4f53_a789_64d173640f23.slice/crio-ec7ff086abefaa128a1341e0520f571cbe80218301a561f76dfe246370c70460 WatchSource:0}: Error finding container ec7ff086abefaa128a1341e0520f571cbe80218301a561f76dfe246370c70460: Status 404 returned error can't find the container with id ec7ff086abefaa128a1341e0520f571cbe80218301a561f76dfe246370c70460
May 28 17:19:18.676835 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:18.676798    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-57584b7c88-llqzq" event={"ID":"31e62ced-916f-4f53-a789-64d173640f23","Type":"ContainerStarted","Data":"ec7ff086abefaa128a1341e0520f571cbe80218301a561f76dfe246370c70460"}
May 28 17:19:19.681033 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:19.680996    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-57584b7c88-llqzq" event={"ID":"31e62ced-916f-4f53-a789-64d173640f23","Type":"ContainerStarted","Data":"542192acb7ee02f21e491ef0985ab05fd6d0a8228584d6236eee6e1ec682ad47"}
May 28 17:19:19.704417 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:19.704371    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-57584b7c88-llqzq" podStartSLOduration=1.470970822 podStartE2EDuration="2.704355676s" podCreationTimestamp="2026-05-28 17:19:17 +0000 UTC" firstStartedPulling="2026-05-28 17:19:18.24551728 +0000 UTC m=+122.485067650" lastFinishedPulling="2026-05-28 17:19:19.478902136 +0000 UTC m=+123.718452504" observedRunningTime="2026-05-28 17:19:19.703905487 +0000 UTC m=+123.943455902" watchObservedRunningTime="2026-05-28 17:19:19.704355676 +0000 UTC m=+123.943906065"
May 28 17:19:26.061241 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:26.061198    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs\") pod \"network-metrics-daemon-7xrml\" (UID: \"1a8e2081-5b89-4115-82b2-d1afbf6f58ed\") " pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:19:26.063764 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:26.063737    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a8e2081-5b89-4115-82b2-d1afbf6f58ed-metrics-certs\") pod \"network-metrics-daemon-7xrml\" (UID: \"1a8e2081-5b89-4115-82b2-d1afbf6f58ed\") " pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:19:26.237594 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:26.237559    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-svsk4\""
May 28 17:19:26.244988 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:26.244965    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xrml"
May 28 17:19:26.392577 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:26.392542    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7xrml"]
May 28 17:19:26.395250 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:19:26.395219    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a8e2081_5b89_4115_82b2_d1afbf6f58ed.slice/crio-5cd81efdc67fcd795afeb8a2c4bac37298fa6e0ea14f1f7a3ba3060029e17722 WatchSource:0}: Error finding container 5cd81efdc67fcd795afeb8a2c4bac37298fa6e0ea14f1f7a3ba3060029e17722: Status 404 returned error can't find the container with id 5cd81efdc67fcd795afeb8a2c4bac37298fa6e0ea14f1f7a3ba3060029e17722
May 28 17:19:26.643818 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:26.643745    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7467f99487-r96cs"
May 28 17:19:26.701746 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:26.701710    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7xrml" event={"ID":"1a8e2081-5b89-4115-82b2-d1afbf6f58ed","Type":"ContainerStarted","Data":"5cd81efdc67fcd795afeb8a2c4bac37298fa6e0ea14f1f7a3ba3060029e17722"}
May 28 17:19:27.706803 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:27.706760    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7xrml" event={"ID":"1a8e2081-5b89-4115-82b2-d1afbf6f58ed","Type":"ContainerStarted","Data":"1c11f20f4a1b5f4d380e4b87c94685b6e1cf9274a7e7331c43f4be5bc15a52f2"}
May 28 17:19:27.706803 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:27.706803    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7xrml" event={"ID":"1a8e2081-5b89-4115-82b2-d1afbf6f58ed","Type":"ContainerStarted","Data":"fbe49ceef0ef25c85dcef7a237f6576755748378b3154b34fee0bfe9681e5f0c"}
May 28 17:19:27.725131 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:27.725078    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7xrml" podStartSLOduration=130.812565657 podStartE2EDuration="2m11.725063672s" podCreationTimestamp="2026-05-28 17:17:16 +0000 UTC" firstStartedPulling="2026-05-28 17:19:26.397155293 +0000 UTC m=+130.636705661" lastFinishedPulling="2026-05-28 17:19:27.309653166 +0000 UTC m=+131.549203676" observedRunningTime="2026-05-28 17:19:27.723938402 +0000 UTC m=+131.963488794" watchObservedRunningTime="2026-05-28 17:19:27.725063672 +0000 UTC m=+131.964614063"
May 28 17:19:34.590629 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:34.590586    2561 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw" podUID="b1fe7206-e183-43be-9c22-27dd1066df34" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 17:19:37.371979 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:37.371928    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9mdhg_7e64cb6c-a574-4272-acc4-5607085c3a2f/serve-healthcheck-canary/0.log"
May 28 17:19:38.115368 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:38.115328    2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:38.115368 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:38.115375    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:44.590426 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:44.590384    2561 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw" podUID="b1fe7206-e183-43be-9c22-27dd1066df34" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 17:19:54.590394 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:54.590354    2561 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw" podUID="b1fe7206-e183-43be-9c22-27dd1066df34" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 17:19:54.590776 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:54.590425    2561 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw"
May 28 17:19:54.591043 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:54.591000    2561 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"4a296dcbfdd408da1726e5be77570c437cfbd41b0395fe8cb6156c442221b875"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw" containerMessage="Container service-proxy failed liveness probe, will be restarted"
May 28 17:19:54.591097 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:54.591081    2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw" podUID="b1fe7206-e183-43be-9c22-27dd1066df34" containerName="service-proxy" containerID="cri-o://4a296dcbfdd408da1726e5be77570c437cfbd41b0395fe8cb6156c442221b875" gracePeriod=30
May 28 17:19:54.783381 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:54.783344    2561 generic.go:358] "Generic (PLEG): container finished" podID="b1fe7206-e183-43be-9c22-27dd1066df34" containerID="4a296dcbfdd408da1726e5be77570c437cfbd41b0395fe8cb6156c442221b875" exitCode=2
May 28 17:19:54.783562 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:54.783408    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw" event={"ID":"b1fe7206-e183-43be-9c22-27dd1066df34","Type":"ContainerDied","Data":"4a296dcbfdd408da1726e5be77570c437cfbd41b0395fe8cb6156c442221b875"}
May 28 17:19:54.783562 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:54.783447    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b56b95fcd-gtrzw" event={"ID":"b1fe7206-e183-43be-9c22-27dd1066df34","Type":"ContainerStarted","Data":"9daa722d2838d8f713877a7b922291699630e774da9b6ea32d64166a5c3beffc"}
May 28 17:19:58.121360 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:58.121325    2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:19:58.125147 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:19:58.125123    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-57584b7c88-llqzq"
May 28 17:22:16.204700 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:16.204657    2561 kubelet.go:1628] "Image garbage collection succeeded"
May 28 17:22:31.167693 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:31.167637    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-trvkj"]
May 28 17:22:31.169779 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:31.169755    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvkj"
May 28 17:22:31.172787 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:31.172767    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 28 17:22:31.180118 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:31.180094    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-trvkj"]
May 28 17:22:31.256886 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:31.256854    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b878c489-e206-490c-954f-fc066b72599b-kubelet-config\") pod \"global-pull-secret-syncer-trvkj\" (UID: \"b878c489-e206-490c-954f-fc066b72599b\") " pod="kube-system/global-pull-secret-syncer-trvkj"
May 28 17:22:31.257067 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:31.256897    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b878c489-e206-490c-954f-fc066b72599b-dbus\") pod \"global-pull-secret-syncer-trvkj\" (UID: \"b878c489-e206-490c-954f-fc066b72599b\") " pod="kube-system/global-pull-secret-syncer-trvkj"
May 28 17:22:31.257067 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:31.256951    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b878c489-e206-490c-954f-fc066b72599b-original-pull-secret\") pod \"global-pull-secret-syncer-trvkj\" (UID: \"b878c489-e206-490c-954f-fc066b72599b\") " pod="kube-system/global-pull-secret-syncer-trvkj"
May 28 17:22:31.357877 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:31.357838    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b878c489-e206-490c-954f-fc066b72599b-kubelet-config\") pod \"global-pull-secret-syncer-trvkj\" (UID: \"b878c489-e206-490c-954f-fc066b72599b\") " pod="kube-system/global-pull-secret-syncer-trvkj"
May 28 17:22:31.358044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:31.357887    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b878c489-e206-490c-954f-fc066b72599b-dbus\") pod \"global-pull-secret-syncer-trvkj\" (UID: \"b878c489-e206-490c-954f-fc066b72599b\") " pod="kube-system/global-pull-secret-syncer-trvkj"
May 28 17:22:31.358044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:31.357977    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b878c489-e206-490c-954f-fc066b72599b-kubelet-config\") pod \"global-pull-secret-syncer-trvkj\" (UID: \"b878c489-e206-490c-954f-fc066b72599b\") " pod="kube-system/global-pull-secret-syncer-trvkj"
May 28 17:22:31.358044 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:31.358012    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b878c489-e206-490c-954f-fc066b72599b-original-pull-secret\") pod \"global-pull-secret-syncer-trvkj\" (UID: \"b878c489-e206-490c-954f-fc066b72599b\") " pod="kube-system/global-pull-secret-syncer-trvkj"
May 28 17:22:31.358152 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:31.358041    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b878c489-e206-490c-954f-fc066b72599b-dbus\") pod \"global-pull-secret-syncer-trvkj\" (UID: \"b878c489-e206-490c-954f-fc066b72599b\") " pod="kube-system/global-pull-secret-syncer-trvkj"
May 28 17:22:31.360368 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:31.360342    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b878c489-e206-490c-954f-fc066b72599b-original-pull-secret\") pod \"global-pull-secret-syncer-trvkj\" (UID: \"b878c489-e206-490c-954f-fc066b72599b\") " pod="kube-system/global-pull-secret-syncer-trvkj"
May 28 17:22:31.479103 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:31.479025    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvkj"
May 28 17:22:31.604376 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:31.604340    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-trvkj"]
May 28 17:22:31.607513 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:22:31.607490    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb878c489_e206_490c_954f_fc066b72599b.slice/crio-e7f844d32ac25504db32647e8fad4c124d315988d7b5b340e4799e63d4781804 WatchSource:0}: Error finding container e7f844d32ac25504db32647e8fad4c124d315988d7b5b340e4799e63d4781804: Status 404 returned error can't find the container with id e7f844d32ac25504db32647e8fad4c124d315988d7b5b340e4799e63d4781804
May 28 17:22:31.609433 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:31.609418    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 17:22:32.172630 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:32.172590    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-trvkj" event={"ID":"b878c489-e206-490c-954f-fc066b72599b","Type":"ContainerStarted","Data":"e7f844d32ac25504db32647e8fad4c124d315988d7b5b340e4799e63d4781804"}
May 28 17:22:37.186421 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:37.186380    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-trvkj" event={"ID":"b878c489-e206-490c-954f-fc066b72599b","Type":"ContainerStarted","Data":"7350ce5d6a5c252d9517c4e241ede275f41ca47598f06820e738de1bf9be5f6a"}
May 28 17:22:37.204028 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:22:37.203972    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-trvkj" podStartSLOduration=1.011379816 podStartE2EDuration="6.203957926s" podCreationTimestamp="2026-05-28 17:22:31 +0000 UTC" firstStartedPulling="2026-05-28 17:22:31.609545954 +0000 UTC m=+315.849096323" lastFinishedPulling="2026-05-28 17:22:36.802124064 +0000 UTC m=+321.041674433" observedRunningTime="2026-05-28 17:22:37.202307018 +0000 UTC m=+321.441857409" watchObservedRunningTime="2026-05-28 17:22:37.203957926 +0000 UTC m=+321.443508316"
May 28 17:23:42.844528 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:42.844494    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-6dsws"]
May 28 17:23:42.846483 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:42.846467    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-6dsws"
May 28 17:23:42.849079 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:42.849053    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
May 28 17:23:42.849223 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:42.849092    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-f994c\""
May 28 17:23:42.850129 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:42.850107    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
May 28 17:23:42.861298 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:42.861274    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-6dsws"]
May 28 17:23:42.970566 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:42.970528    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg77r\" (UniqueName: \"kubernetes.io/projected/7a09bd56-1601-4cb0-9631-d3303fa07c91-kube-api-access-qg77r\") pod \"cert-manager-79c8d999ff-6dsws\" (UID: \"7a09bd56-1601-4cb0-9631-d3303fa07c91\") " pod="cert-manager/cert-manager-79c8d999ff-6dsws"
May 28 17:23:42.970784 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:42.970594    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7a09bd56-1601-4cb0-9631-d3303fa07c91-bound-sa-token\") pod \"cert-manager-79c8d999ff-6dsws\" (UID: \"7a09bd56-1601-4cb0-9631-d3303fa07c91\") " pod="cert-manager/cert-manager-79c8d999ff-6dsws"
May 28 17:23:43.071609 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:43.071575    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qg77r\" (UniqueName: \"kubernetes.io/projected/7a09bd56-1601-4cb0-9631-d3303fa07c91-kube-api-access-qg77r\") pod \"cert-manager-79c8d999ff-6dsws\" (UID: \"7a09bd56-1601-4cb0-9631-d3303fa07c91\") " pod="cert-manager/cert-manager-79c8d999ff-6dsws"
May 28 17:23:43.071803 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:43.071636    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7a09bd56-1601-4cb0-9631-d3303fa07c91-bound-sa-token\") pod \"cert-manager-79c8d999ff-6dsws\" (UID: \"7a09bd56-1601-4cb0-9631-d3303fa07c91\") " pod="cert-manager/cert-manager-79c8d999ff-6dsws"
May 28 17:23:43.081024 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:43.080994    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7a09bd56-1601-4cb0-9631-d3303fa07c91-bound-sa-token\") pod \"cert-manager-79c8d999ff-6dsws\" (UID: \"7a09bd56-1601-4cb0-9631-d3303fa07c91\") " pod="cert-manager/cert-manager-79c8d999ff-6dsws"
May 28 17:23:43.081246 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:43.081223    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg77r\" (UniqueName: \"kubernetes.io/projected/7a09bd56-1601-4cb0-9631-d3303fa07c91-kube-api-access-qg77r\") pod \"cert-manager-79c8d999ff-6dsws\" (UID: \"7a09bd56-1601-4cb0-9631-d3303fa07c91\") " pod="cert-manager/cert-manager-79c8d999ff-6dsws"
May 28 17:23:43.155234 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:43.155143    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-6dsws"
May 28 17:23:43.284236 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:43.284203    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-6dsws"]
May 28 17:23:43.287198 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:23:43.287150    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a09bd56_1601_4cb0_9631_d3303fa07c91.slice/crio-c9a49fa9c367e5389d91d41c88f5729a428914a7035674f36eb0676aac9b3b1e WatchSource:0}: Error finding container c9a49fa9c367e5389d91d41c88f5729a428914a7035674f36eb0676aac9b3b1e: Status 404 returned error can't find the container with id c9a49fa9c367e5389d91d41c88f5729a428914a7035674f36eb0676aac9b3b1e
May 28 17:23:43.347560 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:43.347523    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-6dsws" event={"ID":"7a09bd56-1601-4cb0-9631-d3303fa07c91","Type":"ContainerStarted","Data":"c9a49fa9c367e5389d91d41c88f5729a428914a7035674f36eb0676aac9b3b1e"}
May 28 17:23:46.357229 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:46.357191    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-6dsws" event={"ID":"7a09bd56-1601-4cb0-9631-d3303fa07c91","Type":"ContainerStarted","Data":"f473ec48b4ec706153905bdb3a8188fc376b2f4ecd99d678ca2fbbdfa9730ca6"}
May 28 17:23:46.376942 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:23:46.376894    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-6dsws" podStartSLOduration=2.240335533 podStartE2EDuration="4.376879887s" podCreationTimestamp="2026-05-28 17:23:42 +0000 UTC" firstStartedPulling="2026-05-28 17:23:43.288954013 +0000 UTC m=+387.528504381" lastFinishedPulling="2026-05-28 17:23:45.425498365 +0000 UTC m=+389.665048735" observedRunningTime="2026-05-28 17:23:46.376391504 +0000 UTC m=+390.615941896" watchObservedRunningTime="2026-05-28 17:23:46.376879887 +0000 UTC m=+390.616430278"
May 28 17:24:06.951941 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:06.951908    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx"]
May 28 17:24:06.953967 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:06.953947    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx"
May 28 17:24:06.957202 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:06.957175    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
May 28 17:24:06.957202 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:06.957197    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
May 28 17:24:06.957372 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:06.957276    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
May 28 17:24:06.957557 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:06.957543    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kmhgz\""
May 28 17:24:06.961697 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:06.961659    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
May 28 17:24:07.002116 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:07.002084    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx"]
May 28 17:24:07.055053 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:07.055015    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9904e36d-4e36-4267-9781-a2d6f90c97e1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6795fd7c67-r6plx\" (UID: \"9904e36d-4e36-4267-9781-a2d6f90c97e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx"
May 28 17:24:07.055188 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:07.055060    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsfmg\" (UniqueName: \"kubernetes.io/projected/9904e36d-4e36-4267-9781-a2d6f90c97e1-kube-api-access-wsfmg\") pod \"opendatahub-operator-controller-manager-6795fd7c67-r6plx\" (UID: \"9904e36d-4e36-4267-9781-a2d6f90c97e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx"
May 28 17:24:07.055188 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:07.055172    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9904e36d-4e36-4267-9781-a2d6f90c97e1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6795fd7c67-r6plx\" (UID: \"9904e36d-4e36-4267-9781-a2d6f90c97e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx"
May 28 17:24:07.156560 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:07.156517    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9904e36d-4e36-4267-9781-a2d6f90c97e1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6795fd7c67-r6plx\" (UID: \"9904e36d-4e36-4267-9781-a2d6f90c97e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx"
May 28 17:24:07.156734 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:07.156570    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9904e36d-4e36-4267-9781-a2d6f90c97e1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6795fd7c67-r6plx\" (UID: \"9904e36d-4e36-4267-9781-a2d6f90c97e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx"
May 28 17:24:07.156734 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:07.156611    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wsfmg\" (UniqueName: \"kubernetes.io/projected/9904e36d-4e36-4267-9781-a2d6f90c97e1-kube-api-access-wsfmg\") pod \"opendatahub-operator-controller-manager-6795fd7c67-r6plx\" (UID: \"9904e36d-4e36-4267-9781-a2d6f90c97e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx"
May 28 17:24:07.159227 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:07.159196    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9904e36d-4e36-4267-9781-a2d6f90c97e1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6795fd7c67-r6plx\" (UID: \"9904e36d-4e36-4267-9781-a2d6f90c97e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx"
May 28 17:24:07.159332 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:07.159232    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9904e36d-4e36-4267-9781-a2d6f90c97e1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6795fd7c67-r6plx\" (UID: \"9904e36d-4e36-4267-9781-a2d6f90c97e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx"
May 28 17:24:07.165907 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:07.165886    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsfmg\" (UniqueName: \"kubernetes.io/projected/9904e36d-4e36-4267-9781-a2d6f90c97e1-kube-api-access-wsfmg\") pod \"opendatahub-operator-controller-manager-6795fd7c67-r6plx\" (UID: \"9904e36d-4e36-4267-9781-a2d6f90c97e1\") " pod="opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx"
May 28 17:24:07.264099 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:07.264003    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx"
May 28 17:24:07.392219 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:07.392187    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx"]
May 28 17:24:07.394981 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:24:07.394956    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9904e36d_4e36_4267_9781_a2d6f90c97e1.slice/crio-eaba79fe321c5a7b12977ea9cde0cf2d122c777e15ca2fafc45853ca005e660b WatchSource:0}: Error finding container eaba79fe321c5a7b12977ea9cde0cf2d122c777e15ca2fafc45853ca005e660b: Status 404 returned error can't find the container with id eaba79fe321c5a7b12977ea9cde0cf2d122c777e15ca2fafc45853ca005e660b
May 28 17:24:07.409155 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:07.409130    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx" event={"ID":"9904e36d-4e36-4267-9781-a2d6f90c97e1","Type":"ContainerStarted","Data":"eaba79fe321c5a7b12977ea9cde0cf2d122c777e15ca2fafc45853ca005e660b"}
May 28 17:24:10.419310 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:10.419268    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx" event={"ID":"9904e36d-4e36-4267-9781-a2d6f90c97e1","Type":"ContainerStarted","Data":"0f2b212138c918a9c2ebe0d84cbca3bf60a49b8ebb3f159fe0b350653cab23db"}
May 28 17:24:10.419728 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:10.419401    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx"
May 28 17:24:10.454407 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:10.454347    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx" podStartSLOduration=1.947344943 podStartE2EDuration="4.454332115s" podCreationTimestamp="2026-05-28 17:24:06 +0000 UTC" firstStartedPulling="2026-05-28 17:24:07.396614757 +0000 UTC m=+411.636165125" lastFinishedPulling="2026-05-28 17:24:09.903601914 +0000 UTC m=+414.143152297" observedRunningTime="2026-05-28 17:24:10.451962955 +0000 UTC m=+414.691513357" watchObservedRunningTime="2026-05-28 17:24:10.454332115 +0000 UTC m=+414.693882504"
May 28 17:24:21.425178 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:21.425148    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6795fd7c67-r6plx"
May 28 17:24:24.256042 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.255999    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5454695dcd-kscnq"]
May 28 17:24:24.258847 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.258822    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5454695dcd-kscnq"
May 28 17:24:24.261911 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.261885    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
May 28 17:24:24.262234 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.262216    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
May 28 17:24:24.263373 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.263353    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
May 28 17:24:24.263484 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.263429    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-zgknt\""
May 28 17:24:24.263484 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.263435    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
May 28 17:24:24.269641 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.269617    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5454695dcd-kscnq"]
May 28 17:24:24.279863 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.279836    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l92v\" (UniqueName: \"kubernetes.io/projected/aba9bf3a-7748-495c-a1a2-f8884e127de7-kube-api-access-6l92v\") pod \"kube-auth-proxy-5454695dcd-kscnq\" (UID: \"aba9bf3a-7748-495c-a1a2-f8884e127de7\") " pod="openshift-ingress/kube-auth-proxy-5454695dcd-kscnq"
May 28 17:24:24.280011 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.279915    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aba9bf3a-7748-495c-a1a2-f8884e127de7-tmp\") pod \"kube-auth-proxy-5454695dcd-kscnq\" (UID: \"aba9bf3a-7748-495c-a1a2-f8884e127de7\") " pod="openshift-ingress/kube-auth-proxy-5454695dcd-kscnq"
May 28 17:24:24.280011 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.279961    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aba9bf3a-7748-495c-a1a2-f8884e127de7-tls-certs\") pod \"kube-auth-proxy-5454695dcd-kscnq\" (UID: \"aba9bf3a-7748-495c-a1a2-f8884e127de7\") " pod="openshift-ingress/kube-auth-proxy-5454695dcd-kscnq"
May 28 17:24:24.380594 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.380550    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6l92v\" (UniqueName: \"kubernetes.io/projected/aba9bf3a-7748-495c-a1a2-f8884e127de7-kube-api-access-6l92v\") pod \"kube-auth-proxy-5454695dcd-kscnq\" (UID: \"aba9bf3a-7748-495c-a1a2-f8884e127de7\") " pod="openshift-ingress/kube-auth-proxy-5454695dcd-kscnq"
May 28 17:24:24.380826 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.380611    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aba9bf3a-7748-495c-a1a2-f8884e127de7-tmp\") pod \"kube-auth-proxy-5454695dcd-kscnq\" (UID: \"aba9bf3a-7748-495c-a1a2-f8884e127de7\") " pod="openshift-ingress/kube-auth-proxy-5454695dcd-kscnq"
May 28 17:24:24.380826 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.380655    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aba9bf3a-7748-495c-a1a2-f8884e127de7-tls-certs\") pod \"kube-auth-proxy-5454695dcd-kscnq\" (UID: \"aba9bf3a-7748-495c-a1a2-f8884e127de7\") " pod="openshift-ingress/kube-auth-proxy-5454695dcd-kscnq"
May 28 17:24:24.383060 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.383036    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aba9bf3a-7748-495c-a1a2-f8884e127de7-tmp\") pod \"kube-auth-proxy-5454695dcd-kscnq\" (UID: \"aba9bf3a-7748-495c-a1a2-f8884e127de7\") " pod="openshift-ingress/kube-auth-proxy-5454695dcd-kscnq"
May 28 17:24:24.383283 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.383262    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aba9bf3a-7748-495c-a1a2-f8884e127de7-tls-certs\") pod \"kube-auth-proxy-5454695dcd-kscnq\" (UID: \"aba9bf3a-7748-495c-a1a2-f8884e127de7\") " pod="openshift-ingress/kube-auth-proxy-5454695dcd-kscnq"
May 28 17:24:24.390048 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.390026    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l92v\" (UniqueName: \"kubernetes.io/projected/aba9bf3a-7748-495c-a1a2-f8884e127de7-kube-api-access-6l92v\") pod \"kube-auth-proxy-5454695dcd-kscnq\" (UID: \"aba9bf3a-7748-495c-a1a2-f8884e127de7\") " pod="openshift-ingress/kube-auth-proxy-5454695dcd-kscnq"
May 28 17:24:24.568571 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.568542    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5454695dcd-kscnq"
May 28 17:24:24.696911 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:24.696883    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5454695dcd-kscnq"]
May 28 17:24:24.700600 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:24:24.700574    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaba9bf3a_7748_495c_a1a2_f8884e127de7.slice/crio-82e6ccf1e3079492dbe345c3c40cb8e0e12382c3623f34f00bf67ab748f47913 WatchSource:0}: Error finding container 82e6ccf1e3079492dbe345c3c40cb8e0e12382c3623f34f00bf67ab748f47913: Status 404 returned error can't find the container with id 82e6ccf1e3079492dbe345c3c40cb8e0e12382c3623f34f00bf67ab748f47913
May 28 17:24:25.463309 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:25.463256    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5454695dcd-kscnq" event={"ID":"aba9bf3a-7748-495c-a1a2-f8884e127de7","Type":"ContainerStarted","Data":"82e6ccf1e3079492dbe345c3c40cb8e0e12382c3623f34f00bf67ab748f47913"}
May 28 17:24:27.261323 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:27.261290    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-ghngv"]
May 28 17:24:27.263077 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:27.263056    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-ghngv"
May 28 17:24:27.265632 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:27.265609    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
May 28 17:24:27.265771 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:27.265651    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-jq9zg\""
May 28 17:24:27.273291 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:27.273263    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-ghngv"]
May 28 17:24:27.309423 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:27.309392    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzg6k\" (UniqueName: \"kubernetes.io/projected/b596c9a0-b864-48a9-8e2d-50636502112d-kube-api-access-vzg6k\") pod \"odh-model-controller-858dbf95b8-ghngv\" (UID: \"b596c9a0-b864-48a9-8e2d-50636502112d\") " pod="opendatahub/odh-model-controller-858dbf95b8-ghngv"
May 28 17:24:27.309544 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:27.309435    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b596c9a0-b864-48a9-8e2d-50636502112d-cert\") pod \"odh-model-controller-858dbf95b8-ghngv\" (UID: \"b596c9a0-b864-48a9-8e2d-50636502112d\") " pod="opendatahub/odh-model-controller-858dbf95b8-ghngv"
May 28 17:24:27.410549 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:27.410507    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vzg6k\" (UniqueName: \"kubernetes.io/projected/b596c9a0-b864-48a9-8e2d-50636502112d-kube-api-access-vzg6k\") pod \"odh-model-controller-858dbf95b8-ghngv\" (UID: \"b596c9a0-b864-48a9-8e2d-50636502112d\") " pod="opendatahub/odh-model-controller-858dbf95b8-ghngv"
May 28 17:24:27.410549 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:27.410543    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b596c9a0-b864-48a9-8e2d-50636502112d-cert\") pod \"odh-model-controller-858dbf95b8-ghngv\" (UID: \"b596c9a0-b864-48a9-8e2d-50636502112d\") " pod="opendatahub/odh-model-controller-858dbf95b8-ghngv"
May 28 17:24:27.410829 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:24:27.410741    2561 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
May 28 17:24:27.410829 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:24:27.410792    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b596c9a0-b864-48a9-8e2d-50636502112d-cert podName:b596c9a0-b864-48a9-8e2d-50636502112d nodeName:}" failed. No retries permitted until 2026-05-28 17:24:27.910775959 +0000 UTC m=+432.150326327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b596c9a0-b864-48a9-8e2d-50636502112d-cert") pod "odh-model-controller-858dbf95b8-ghngv" (UID: "b596c9a0-b864-48a9-8e2d-50636502112d") : secret "odh-model-controller-webhook-cert" not found
May 28 17:24:27.421086 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:27.421053    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzg6k\" (UniqueName: \"kubernetes.io/projected/b596c9a0-b864-48a9-8e2d-50636502112d-kube-api-access-vzg6k\") pod \"odh-model-controller-858dbf95b8-ghngv\" (UID: \"b596c9a0-b864-48a9-8e2d-50636502112d\") " pod="opendatahub/odh-model-controller-858dbf95b8-ghngv"
May 28 17:24:27.470131 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:27.470097    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5454695dcd-kscnq" event={"ID":"aba9bf3a-7748-495c-a1a2-f8884e127de7","Type":"ContainerStarted","Data":"a2a021cfebc0b2cba1fd6c676ea900138fa844417027c7e65ad01a3a7b12e3de"}
May 28 17:24:27.488275 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:27.488223    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5454695dcd-kscnq" podStartSLOduration=0.90327076 podStartE2EDuration="3.488209653s" podCreationTimestamp="2026-05-28 17:24:24 +0000 UTC" firstStartedPulling="2026-05-28 17:24:24.702453738 +0000 UTC m=+428.942004108" lastFinishedPulling="2026-05-28 17:24:27.287392626 +0000 UTC m=+431.526943001" observedRunningTime="2026-05-28 17:24:27.487701911 +0000 UTC m=+431.727252294" watchObservedRunningTime="2026-05-28 17:24:27.488209653 +0000 UTC m=+431.727760044"
May 28 17:24:27.914639 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:27.914581    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b596c9a0-b864-48a9-8e2d-50636502112d-cert\") pod \"odh-model-controller-858dbf95b8-ghngv\" (UID: \"b596c9a0-b864-48a9-8e2d-50636502112d\") " pod="opendatahub/odh-model-controller-858dbf95b8-ghngv"
May 28 17:24:27.914848 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:24:27.914752    2561 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
May 28 17:24:27.914848 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:24:27.914816    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b596c9a0-b864-48a9-8e2d-50636502112d-cert podName:b596c9a0-b864-48a9-8e2d-50636502112d nodeName:}" failed. No retries permitted until 2026-05-28 17:24:28.914800686 +0000 UTC m=+433.154351053 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b596c9a0-b864-48a9-8e2d-50636502112d-cert") pod "odh-model-controller-858dbf95b8-ghngv" (UID: "b596c9a0-b864-48a9-8e2d-50636502112d") : secret "odh-model-controller-webhook-cert" not found
May 28 17:24:28.923424 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:28.923338    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b596c9a0-b864-48a9-8e2d-50636502112d-cert\") pod \"odh-model-controller-858dbf95b8-ghngv\" (UID: \"b596c9a0-b864-48a9-8e2d-50636502112d\") " pod="opendatahub/odh-model-controller-858dbf95b8-ghngv"
May 28 17:24:28.926096 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:28.926072    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b596c9a0-b864-48a9-8e2d-50636502112d-cert\") pod \"odh-model-controller-858dbf95b8-ghngv\" (UID: \"b596c9a0-b864-48a9-8e2d-50636502112d\") " pod="opendatahub/odh-model-controller-858dbf95b8-ghngv"
May 28 17:24:29.074347 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:29.074308    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-ghngv"
May 28 17:24:29.201201 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:29.201168    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-ghngv"]
May 28 17:24:29.203778 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:24:29.203748    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb596c9a0_b864_48a9_8e2d_50636502112d.slice/crio-c6da4e085640bee1684504a5e485d98a3bc4e8108f36b8df07cfc29e81b89f79 WatchSource:0}: Error finding container c6da4e085640bee1684504a5e485d98a3bc4e8108f36b8df07cfc29e81b89f79: Status 404 returned error can't find the container with id c6da4e085640bee1684504a5e485d98a3bc4e8108f36b8df07cfc29e81b89f79
May 28 17:24:29.477073 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:29.476996    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-ghngv" event={"ID":"b596c9a0-b864-48a9-8e2d-50636502112d","Type":"ContainerStarted","Data":"c6da4e085640bee1684504a5e485d98a3bc4e8108f36b8df07cfc29e81b89f79"}
May 28 17:24:32.489362 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:32.489325    2561 generic.go:358] "Generic (PLEG): container finished" podID="b596c9a0-b864-48a9-8e2d-50636502112d" containerID="2c92327599338d57cca466e8dd76f421a28b9bd3c4d0d50320fb4d644336b020" exitCode=1
May 28 17:24:32.489762 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:32.489396    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-ghngv" event={"ID":"b596c9a0-b864-48a9-8e2d-50636502112d","Type":"ContainerDied","Data":"2c92327599338d57cca466e8dd76f421a28b9bd3c4d0d50320fb4d644336b020"}
May 28 17:24:32.489762 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:32.489617    2561 scope.go:117] "RemoveContainer" containerID="2c92327599338d57cca466e8dd76f421a28b9bd3c4d0d50320fb4d644336b020"
May 28 17:24:33.247324 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:33.247287    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-jbd9m"]
May 28 17:24:33.249381 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:33.249362    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-jbd9m"
May 28 17:24:33.252268 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:33.252238    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
May 28 17:24:33.252404 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:33.252279    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-fc94t\""
May 28 17:24:33.259560 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:33.259526    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-jbd9m"]
May 28 17:24:33.359940 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:33.359900    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx4fs\" (UniqueName: \"kubernetes.io/projected/c51e64e5-6eac-42a5-9533-cd8c059a542b-kube-api-access-mx4fs\") pod \"kserve-controller-manager-856948b99f-jbd9m\" (UID: \"c51e64e5-6eac-42a5-9533-cd8c059a542b\") " pod="opendatahub/kserve-controller-manager-856948b99f-jbd9m"
May 28 17:24:33.360126 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:33.359974    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c51e64e5-6eac-42a5-9533-cd8c059a542b-cert\") pod \"kserve-controller-manager-856948b99f-jbd9m\" (UID: \"c51e64e5-6eac-42a5-9533-cd8c059a542b\") " pod="opendatahub/kserve-controller-manager-856948b99f-jbd9m"
May 28 17:24:33.460860 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:33.460819    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c51e64e5-6eac-42a5-9533-cd8c059a542b-cert\") pod \"kserve-controller-manager-856948b99f-jbd9m\" (UID: \"c51e64e5-6eac-42a5-9533-cd8c059a542b\") " pod="opendatahub/kserve-controller-manager-856948b99f-jbd9m"
May 28 17:24:33.461047 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:33.460952    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mx4fs\" (UniqueName: \"kubernetes.io/projected/c51e64e5-6eac-42a5-9533-cd8c059a542b-kube-api-access-mx4fs\") pod \"kserve-controller-manager-856948b99f-jbd9m\" (UID: \"c51e64e5-6eac-42a5-9533-cd8c059a542b\") " pod="opendatahub/kserve-controller-manager-856948b99f-jbd9m"
May 28 17:24:33.461047 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:24:33.460978    2561 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
May 28 17:24:33.461047 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:24:33.461044    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c51e64e5-6eac-42a5-9533-cd8c059a542b-cert podName:c51e64e5-6eac-42a5-9533-cd8c059a542b nodeName:}" failed. No retries permitted until 2026-05-28 17:24:33.9610229 +0000 UTC m=+438.200573268 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c51e64e5-6eac-42a5-9533-cd8c059a542b-cert") pod "kserve-controller-manager-856948b99f-jbd9m" (UID: "c51e64e5-6eac-42a5-9533-cd8c059a542b") : secret "kserve-webhook-server-cert" not found
May 28 17:24:33.473906 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:33.473881    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx4fs\" (UniqueName: \"kubernetes.io/projected/c51e64e5-6eac-42a5-9533-cd8c059a542b-kube-api-access-mx4fs\") pod \"kserve-controller-manager-856948b99f-jbd9m\" (UID: \"c51e64e5-6eac-42a5-9533-cd8c059a542b\") " pod="opendatahub/kserve-controller-manager-856948b99f-jbd9m"
May 28 17:24:33.493810 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:33.493778    2561 generic.go:358] "Generic (PLEG): container finished" podID="b596c9a0-b864-48a9-8e2d-50636502112d" containerID="ccf1513dded650a07a90a7d67c8b07d6f766e4296f305a8a6829363a249ed6f6" exitCode=1
May 28 17:24:33.494217 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:33.493826    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-ghngv" event={"ID":"b596c9a0-b864-48a9-8e2d-50636502112d","Type":"ContainerDied","Data":"ccf1513dded650a07a90a7d67c8b07d6f766e4296f305a8a6829363a249ed6f6"}
May 28 17:24:33.494217 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:33.493853    2561 scope.go:117] "RemoveContainer" containerID="2c92327599338d57cca466e8dd76f421a28b9bd3c4d0d50320fb4d644336b020"
May 28 17:24:33.494217 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:33.494097    2561 scope.go:117] "RemoveContainer" containerID="ccf1513dded650a07a90a7d67c8b07d6f766e4296f305a8a6829363a249ed6f6"
May 28 17:24:33.494339 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:24:33.494300    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-ghngv_opendatahub(b596c9a0-b864-48a9-8e2d-50636502112d)\"" pod="opendatahub/odh-model-controller-858dbf95b8-ghngv" podUID="b596c9a0-b864-48a9-8e2d-50636502112d"
May 28 17:24:33.965357 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:33.965318    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c51e64e5-6eac-42a5-9533-cd8c059a542b-cert\") pod \"kserve-controller-manager-856948b99f-jbd9m\" (UID: \"c51e64e5-6eac-42a5-9533-cd8c059a542b\") " pod="opendatahub/kserve-controller-manager-856948b99f-jbd9m"
May 28 17:24:33.967897 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:33.967874    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c51e64e5-6eac-42a5-9533-cd8c059a542b-cert\") pod \"kserve-controller-manager-856948b99f-jbd9m\" (UID: \"c51e64e5-6eac-42a5-9533-cd8c059a542b\") " pod="opendatahub/kserve-controller-manager-856948b99f-jbd9m"
May 28 17:24:34.161475 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:34.161434    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-jbd9m"
May 28 17:24:34.296490 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:34.296455    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-jbd9m"]
May 28 17:24:34.299508 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:24:34.299475    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc51e64e5_6eac_42a5_9533_cd8c059a542b.slice/crio-3090e03987a5cc9f779ad77c9c237746278abbad3f4038d2610669b5a4048d7f WatchSource:0}: Error finding container 3090e03987a5cc9f779ad77c9c237746278abbad3f4038d2610669b5a4048d7f: Status 404 returned error can't find the container with id 3090e03987a5cc9f779ad77c9c237746278abbad3f4038d2610669b5a4048d7f
May 28 17:24:34.498438 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:34.498354    2561 scope.go:117] "RemoveContainer" containerID="ccf1513dded650a07a90a7d67c8b07d6f766e4296f305a8a6829363a249ed6f6"
May 28 17:24:34.498938 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:24:34.498579    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-ghngv_opendatahub(b596c9a0-b864-48a9-8e2d-50636502112d)\"" pod="opendatahub/odh-model-controller-858dbf95b8-ghngv" podUID="b596c9a0-b864-48a9-8e2d-50636502112d"
May 28 17:24:34.499217 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:34.499193    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-jbd9m" event={"ID":"c51e64e5-6eac-42a5-9533-cd8c059a542b","Type":"ContainerStarted","Data":"3090e03987a5cc9f779ad77c9c237746278abbad3f4038d2610669b5a4048d7f"}
May 28 17:24:37.509009 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:37.508970    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-jbd9m" event={"ID":"c51e64e5-6eac-42a5-9533-cd8c059a542b","Type":"ContainerStarted","Data":"3a8db9c68b6ff095598179181321905e3d2e58dcfb12b3183cff85bda6415dda"}
May 28 17:24:37.509413 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:37.509123    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-856948b99f-jbd9m"
May 28 17:24:37.536659 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:37.536604    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-856948b99f-jbd9m" podStartSLOduration=2.199291818 podStartE2EDuration="4.536588941s" podCreationTimestamp="2026-05-28 17:24:33 +0000 UTC" firstStartedPulling="2026-05-28 17:24:34.301107638 +0000 UTC m=+438.540658019" lastFinishedPulling="2026-05-28 17:24:36.638404775 +0000 UTC m=+440.877955142" observedRunningTime="2026-05-28 17:24:37.536119633 +0000 UTC m=+441.775670024" watchObservedRunningTime="2026-05-28 17:24:37.536588941 +0000 UTC m=+441.776139331"
May 28 17:24:39.036820 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.036784    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-bvp5c"]
May 28 17:24:39.039268 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.039246    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-bvp5c"
May 28 17:24:39.045001 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.044976    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-gwgzr\""
May 28 17:24:39.045461 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.045429    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\""
May 28 17:24:39.045937 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.045796    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\""
May 28 17:24:39.059606 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.059586    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-bvp5c"]
May 28 17:24:39.075410 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.075389    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-ghngv"
May 28 17:24:39.075736 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.075723    2561 scope.go:117] "RemoveContainer" containerID="ccf1513dded650a07a90a7d67c8b07d6f766e4296f305a8a6829363a249ed6f6"
May 28 17:24:39.075894 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:24:39.075879    2561 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-ghngv_opendatahub(b596c9a0-b864-48a9-8e2d-50636502112d)\"" pod="opendatahub/odh-model-controller-858dbf95b8-ghngv" podUID="b596c9a0-b864-48a9-8e2d-50636502112d"
May 28 17:24:39.105997 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.105972    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/321c817a-fe04-43e1-9a68-e12caf058805-operator-config\") pod \"servicemesh-operator3-55f49c5f94-bvp5c\" (UID: \"321c817a-fe04-43e1-9a68-e12caf058805\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-bvp5c"
May 28 17:24:39.106149 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.106006    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjr4s\" (UniqueName: \"kubernetes.io/projected/321c817a-fe04-43e1-9a68-e12caf058805-kube-api-access-bjr4s\") pod \"servicemesh-operator3-55f49c5f94-bvp5c\" (UID: \"321c817a-fe04-43e1-9a68-e12caf058805\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-bvp5c"
May 28 17:24:39.206802 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.206763    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/321c817a-fe04-43e1-9a68-e12caf058805-operator-config\") pod \"servicemesh-operator3-55f49c5f94-bvp5c\" (UID: \"321c817a-fe04-43e1-9a68-e12caf058805\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-bvp5c"
May 28 17:24:39.206802 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.206806    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bjr4s\" (UniqueName: \"kubernetes.io/projected/321c817a-fe04-43e1-9a68-e12caf058805-kube-api-access-bjr4s\") pod \"servicemesh-operator3-55f49c5f94-bvp5c\" (UID: \"321c817a-fe04-43e1-9a68-e12caf058805\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-bvp5c"
May 28 17:24:39.209465 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.209443    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/321c817a-fe04-43e1-9a68-e12caf058805-operator-config\") pod \"servicemesh-operator3-55f49c5f94-bvp5c\" (UID: \"321c817a-fe04-43e1-9a68-e12caf058805\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-bvp5c"
May 28 17:24:39.223347 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.223321    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjr4s\" (UniqueName: \"kubernetes.io/projected/321c817a-fe04-43e1-9a68-e12caf058805-kube-api-access-bjr4s\") pod \"servicemesh-operator3-55f49c5f94-bvp5c\" (UID: \"321c817a-fe04-43e1-9a68-e12caf058805\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-bvp5c"
May 28 17:24:39.348783 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.348745    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-bvp5c"
May 28 17:24:39.492706 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.492662    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-bvp5c"]
May 28 17:24:39.495524 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:24:39.495493    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod321c817a_fe04_43e1_9a68_e12caf058805.slice/crio-af7f3f6a2f20e5154e19e3c4c4090bffd28fea6f81447b3f5e69c8fbb3c8d2b9 WatchSource:0}: Error finding container af7f3f6a2f20e5154e19e3c4c4090bffd28fea6f81447b3f5e69c8fbb3c8d2b9: Status 404 returned error can't find the container with id af7f3f6a2f20e5154e19e3c4c4090bffd28fea6f81447b3f5e69c8fbb3c8d2b9
May 28 17:24:39.520484 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:39.520442    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-bvp5c" event={"ID":"321c817a-fe04-43e1-9a68-e12caf058805","Type":"ContainerStarted","Data":"af7f3f6a2f20e5154e19e3c4c4090bffd28fea6f81447b3f5e69c8fbb3c8d2b9"}
May 28 17:24:42.531169 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:42.531130    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-bvp5c" event={"ID":"321c817a-fe04-43e1-9a68-e12caf058805","Type":"ContainerStarted","Data":"47bb287995cceb9b6dadc4426e66ddea635e6343ed1decfbefd139e16234d485"}
May 28 17:24:42.531579 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:42.531238    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-bvp5c"
May 28 17:24:42.555624 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:42.555565    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-bvp5c" podStartSLOduration=0.716868152 podStartE2EDuration="3.55553049s" podCreationTimestamp="2026-05-28 17:24:39 +0000 UTC" firstStartedPulling="2026-05-28 17:24:39.498117495 +0000 UTC m=+443.737667866" lastFinishedPulling="2026-05-28 17:24:42.336779826 +0000 UTC m=+446.576330204" observedRunningTime="2026-05-28 17:24:42.552577711 +0000 UTC m=+446.792128112" watchObservedRunningTime="2026-05-28 17:24:42.55553049 +0000 UTC m=+446.795080882"
May 28 17:24:49.075006 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:49.074972    2561 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-858dbf95b8-ghngv"
May 28 17:24:49.075385 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:49.075325    2561 scope.go:117] "RemoveContainer" containerID="ccf1513dded650a07a90a7d67c8b07d6f766e4296f305a8a6829363a249ed6f6"
May 28 17:24:49.554760 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:49.554724    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-ghngv" event={"ID":"b596c9a0-b864-48a9-8e2d-50636502112d","Type":"ContainerStarted","Data":"c5b049734a9a2890451137583b62260494b5fe2a851b247fb7a99a3d9ae06750"}
May 28 17:24:49.554951 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:49.554930    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-ghngv"
May 28 17:24:49.574092 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:49.574039    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-858dbf95b8-ghngv" podStartSLOduration=2.398855499 podStartE2EDuration="22.574024717s" podCreationTimestamp="2026-05-28 17:24:27 +0000 UTC" firstStartedPulling="2026-05-28 17:24:29.205553103 +0000 UTC m=+433.445103470" lastFinishedPulling="2026-05-28 17:24:49.380722316 +0000 UTC m=+453.620272688" observedRunningTime="2026-05-28 17:24:49.572381375 +0000 UTC m=+453.811931778" watchObservedRunningTime="2026-05-28 17:24:49.574024717 +0000 UTC m=+453.813575107"
May 28 17:24:53.537243 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.537205    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-bvp5c"
May 28 17:24:53.649701 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.649636    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"]
May 28 17:24:53.652939 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.652916    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.655634 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.655613    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
May 28 17:24:53.656153 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.656135    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gcr7g\""
May 28 17:24:53.656375 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.656357    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
May 28 17:24:53.656437 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.656422    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
May 28 17:24:53.656477 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.656422    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
May 28 17:24:53.676460 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.676421    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"]
May 28 17:24:53.726285 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.726249    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6c2939c-7bbc-407d-a977-afac48c78115-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.726468 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.726308    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6c2939c-7bbc-407d-a977-afac48c78115-istio-token\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.726468 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.726382    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6c2939c-7bbc-407d-a977-afac48c78115-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.726468 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.726434    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6c2939c-7bbc-407d-a977-afac48c78115-cacerts\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.726570 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.726467    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtt5b\" (UniqueName: \"kubernetes.io/projected/f6c2939c-7bbc-407d-a977-afac48c78115-kube-api-access-rtt5b\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.726570 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.726504    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6c2939c-7bbc-407d-a977-afac48c78115-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.726570 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.726537    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6c2939c-7bbc-407d-a977-afac48c78115-local-certs\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.827617 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.827577    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6c2939c-7bbc-407d-a977-afac48c78115-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.827811 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.827631    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6c2939c-7bbc-407d-a977-afac48c78115-local-certs\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.827811 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.827714    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6c2939c-7bbc-407d-a977-afac48c78115-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.827811 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.827789    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6c2939c-7bbc-407d-a977-afac48c78115-istio-token\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.827960 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.827842    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6c2939c-7bbc-407d-a977-afac48c78115-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.827960 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.827884    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6c2939c-7bbc-407d-a977-afac48c78115-cacerts\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.827960 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.827928    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rtt5b\" (UniqueName: \"kubernetes.io/projected/f6c2939c-7bbc-407d-a977-afac48c78115-kube-api-access-rtt5b\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.828657 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.828633    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6c2939c-7bbc-407d-a977-afac48c78115-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.830575 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.830546    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6c2939c-7bbc-407d-a977-afac48c78115-local-certs\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.830862 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.830843    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6c2939c-7bbc-407d-a977-afac48c78115-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.830923 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.830903    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6c2939c-7bbc-407d-a977-afac48c78115-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.830961 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.830924    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6c2939c-7bbc-407d-a977-afac48c78115-cacerts\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.838921 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.838895    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6c2939c-7bbc-407d-a977-afac48c78115-istio-token\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.839049 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.838959    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtt5b\" (UniqueName: \"kubernetes.io/projected/f6c2939c-7bbc-407d-a977-afac48c78115-kube-api-access-rtt5b\") pod \"istiod-openshift-gateway-798958bb55-4c84k\" (UID: \"f6c2939c-7bbc-407d-a977-afac48c78115\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:53.962364 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:53.962324    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:54.125180 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:54.124750    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"]
May 28 17:24:54.127875 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:24:54.127840    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6c2939c_7bbc_407d_a977_afac48c78115.slice/crio-a0e73a65f4f620ce5c5f3ad7cc492257b7103eaa758ec13b7028496670af1c4d WatchSource:0}: Error finding container a0e73a65f4f620ce5c5f3ad7cc492257b7103eaa758ec13b7028496670af1c4d: Status 404 returned error can't find the container with id a0e73a65f4f620ce5c5f3ad7cc492257b7103eaa758ec13b7028496670af1c4d
May 28 17:24:54.571468 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:54.571428    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k" event={"ID":"f6c2939c-7bbc-407d-a977-afac48c78115","Type":"ContainerStarted","Data":"a0e73a65f4f620ce5c5f3ad7cc492257b7103eaa758ec13b7028496670af1c4d"}
May 28 17:24:56.792751 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:56.792695    2561 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
May 28 17:24:56.793081 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:56.792779    2561 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
May 28 17:24:57.584729 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:57.584670    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k" event={"ID":"f6c2939c-7bbc-407d-a977-afac48c78115","Type":"ContainerStarted","Data":"634691c9354e1111b1bc65e1bd3432dc6bf0dc0786985942c0ed1b9a19a1134a"}
May 28 17:24:57.584938 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:57.584913    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:57.586459 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:57.586431    2561 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-4c84k container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
May 28 17:24:57.586609 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:57.586475    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k" podUID="f6c2939c-7bbc-407d-a977-afac48c78115" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 28 17:24:58.588695 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:58.588644    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k"
May 28 17:24:58.613266 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:24:58.613214    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-4c84k" podStartSLOduration=2.950869761 podStartE2EDuration="5.613198944s" podCreationTimestamp="2026-05-28 17:24:53 +0000 UTC" firstStartedPulling="2026-05-28 17:24:54.130100629 +0000 UTC m=+458.369650998" lastFinishedPulling="2026-05-28 17:24:56.792429812 +0000 UTC m=+461.031980181" observedRunningTime="2026-05-28 17:24:57.61003504 +0000 UTC m=+461.849585441" watchObservedRunningTime="2026-05-28 17:24:58.613198944 +0000 UTC m=+462.852749335"
May 28 17:25:00.560839 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:00.560810    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-858dbf95b8-ghngv"
May 28 17:25:08.517564 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:08.517533    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-856948b99f-jbd9m"
May 28 17:25:24.469710 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:24.469658    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mtgb7"]
May 28 17:25:24.483515 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:24.483486    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mtgb7"
May 28 17:25:24.483515 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:24.483503    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mtgb7"]
May 28 17:25:24.486367 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:24.486343    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
May 28 17:25:24.487597 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:24.487575    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-9gqf8\""
May 28 17:25:24.487740 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:24.487602    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
May 28 17:25:24.586993 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:24.586952    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzdv5\" (UniqueName: \"kubernetes.io/projected/31c9a1ab-eb91-4e29-b1eb-c81d58821f46-kube-api-access-gzdv5\") pod \"kuadrant-operator-catalog-mtgb7\" (UID: \"31c9a1ab-eb91-4e29-b1eb-c81d58821f46\") " pod="kuadrant-system/kuadrant-operator-catalog-mtgb7"
May 28 17:25:24.687653 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:24.687616    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gzdv5\" (UniqueName: \"kubernetes.io/projected/31c9a1ab-eb91-4e29-b1eb-c81d58821f46-kube-api-access-gzdv5\") pod \"kuadrant-operator-catalog-mtgb7\" (UID: \"31c9a1ab-eb91-4e29-b1eb-c81d58821f46\") " pod="kuadrant-system/kuadrant-operator-catalog-mtgb7"
May 28 17:25:24.697382 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:24.697347    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzdv5\" (UniqueName: \"kubernetes.io/projected/31c9a1ab-eb91-4e29-b1eb-c81d58821f46-kube-api-access-gzdv5\") pod \"kuadrant-operator-catalog-mtgb7\" (UID: \"31c9a1ab-eb91-4e29-b1eb-c81d58821f46\") " pod="kuadrant-system/kuadrant-operator-catalog-mtgb7"
May 28 17:25:24.794768 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:24.794654    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mtgb7"
May 28 17:25:24.828534 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:24.828497    2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mtgb7"]
May 28 17:25:24.926158 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:24.926127    2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mtgb7"]
May 28 17:25:24.929132 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:25:24.929104    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31c9a1ab_eb91_4e29_b1eb_c81d58821f46.slice/crio-dde5f447f5eb4e6e3622217d11456761a40e94afad8ba3e5c4b7b834cff59d33 WatchSource:0}: Error finding container dde5f447f5eb4e6e3622217d11456761a40e94afad8ba3e5c4b7b834cff59d33: Status 404 returned error can't find the container with id dde5f447f5eb4e6e3622217d11456761a40e94afad8ba3e5c4b7b834cff59d33
May 28 17:25:25.038108 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:25.038070    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d52wt"]
May 28 17:25:25.042516 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:25.042499    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d52wt"
May 28 17:25:25.048247 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:25.048190    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d52wt"]
May 28 17:25:25.090786 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:25.090756    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7m8g\" (UniqueName: \"kubernetes.io/projected/f5ee3f0c-e6c4-4f1b-9876-d3dd8ab5c373-kube-api-access-x7m8g\") pod \"kuadrant-operator-catalog-d52wt\" (UID: \"f5ee3f0c-e6c4-4f1b-9876-d3dd8ab5c373\") " pod="kuadrant-system/kuadrant-operator-catalog-d52wt"
May 28 17:25:25.191843 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:25.191806    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x7m8g\" (UniqueName: \"kubernetes.io/projected/f5ee3f0c-e6c4-4f1b-9876-d3dd8ab5c373-kube-api-access-x7m8g\") pod \"kuadrant-operator-catalog-d52wt\" (UID: \"f5ee3f0c-e6c4-4f1b-9876-d3dd8ab5c373\") " pod="kuadrant-system/kuadrant-operator-catalog-d52wt"
May 28 17:25:25.200748 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:25.200718    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7m8g\" (UniqueName: \"kubernetes.io/projected/f5ee3f0c-e6c4-4f1b-9876-d3dd8ab5c373-kube-api-access-x7m8g\") pod \"kuadrant-operator-catalog-d52wt\" (UID: \"f5ee3f0c-e6c4-4f1b-9876-d3dd8ab5c373\") " pod="kuadrant-system/kuadrant-operator-catalog-d52wt"
May 28 17:25:25.352345 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:25.352313    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d52wt"
May 28 17:25:25.510609 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:25.510583    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d52wt"]
May 28 17:25:25.517436 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:25:25.517410    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ee3f0c_e6c4_4f1b_9876_d3dd8ab5c373.slice/crio-bc82426e48f267b3403feafc4e0702a77d57aa52980b6dcc7d513eb32a9278a6 WatchSource:0}: Error finding container bc82426e48f267b3403feafc4e0702a77d57aa52980b6dcc7d513eb32a9278a6: Status 404 returned error can't find the container with id bc82426e48f267b3403feafc4e0702a77d57aa52980b6dcc7d513eb32a9278a6
May 28 17:25:25.670891 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:25.670800    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d52wt" event={"ID":"f5ee3f0c-e6c4-4f1b-9876-d3dd8ab5c373","Type":"ContainerStarted","Data":"bc82426e48f267b3403feafc4e0702a77d57aa52980b6dcc7d513eb32a9278a6"}
May 28 17:25:25.671974 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:25.671932    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mtgb7" event={"ID":"31c9a1ab-eb91-4e29-b1eb-c81d58821f46","Type":"ContainerStarted","Data":"dde5f447f5eb4e6e3622217d11456761a40e94afad8ba3e5c4b7b834cff59d33"}
May 28 17:25:27.680393 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:27.680360    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d52wt" event={"ID":"f5ee3f0c-e6c4-4f1b-9876-d3dd8ab5c373","Type":"ContainerStarted","Data":"4432d19b20e91baaad99b8dade6940d4535722cc0536001eeac7a124d459ccf2"}
May 28 17:25:27.681598 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:27.681574    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mtgb7" event={"ID":"31c9a1ab-eb91-4e29-b1eb-c81d58821f46","Type":"ContainerStarted","Data":"523f43306ad4481ffd30fdaa4c481ea390c6edf209ea997a1b21bbe5e6d4e420"}
May 28 17:25:27.681719 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:27.681629    2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-mtgb7" podUID="31c9a1ab-eb91-4e29-b1eb-c81d58821f46" containerName="registry-server" containerID="cri-o://523f43306ad4481ffd30fdaa4c481ea390c6edf209ea997a1b21bbe5e6d4e420" gracePeriod=2
May 28 17:25:27.699254 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:27.699217    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-d52wt" podStartSLOduration=1.232902165 podStartE2EDuration="2.699203631s" podCreationTimestamp="2026-05-28 17:25:25 +0000 UTC" firstStartedPulling="2026-05-28 17:25:25.518719304 +0000 UTC m=+489.758269673" lastFinishedPulling="2026-05-28 17:25:26.98502077 +0000 UTC m=+491.224571139" observedRunningTime="2026-05-28 17:25:27.698372914 +0000 UTC m=+491.937923303" watchObservedRunningTime="2026-05-28 17:25:27.699203631 +0000 UTC m=+491.938754020"
May 28 17:25:27.714905 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:27.714866    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-mtgb7" podStartSLOduration=1.663034574 podStartE2EDuration="3.714855206s" podCreationTimestamp="2026-05-28 17:25:24 +0000 UTC" firstStartedPulling="2026-05-28 17:25:24.930402753 +0000 UTC m=+489.169953121" lastFinishedPulling="2026-05-28 17:25:26.982223385 +0000 UTC m=+491.221773753" observedRunningTime="2026-05-28 17:25:27.713307533 +0000 UTC m=+491.952857918" watchObservedRunningTime="2026-05-28 17:25:27.714855206 +0000 UTC m=+491.954405596"
May 28 17:25:27.921628 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:27.921603    2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mtgb7"
May 28 17:25:28.015545 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:28.015433    2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzdv5\" (UniqueName: \"kubernetes.io/projected/31c9a1ab-eb91-4e29-b1eb-c81d58821f46-kube-api-access-gzdv5\") pod \"31c9a1ab-eb91-4e29-b1eb-c81d58821f46\" (UID: \"31c9a1ab-eb91-4e29-b1eb-c81d58821f46\") "
May 28 17:25:28.017786 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:28.017749    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31c9a1ab-eb91-4e29-b1eb-c81d58821f46-kube-api-access-gzdv5" (OuterVolumeSpecName: "kube-api-access-gzdv5") pod "31c9a1ab-eb91-4e29-b1eb-c81d58821f46" (UID: "31c9a1ab-eb91-4e29-b1eb-c81d58821f46"). InnerVolumeSpecName "kube-api-access-gzdv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:25:28.116129 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:28.116089    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzdv5\" (UniqueName: \"kubernetes.io/projected/31c9a1ab-eb91-4e29-b1eb-c81d58821f46-kube-api-access-gzdv5\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\""
May 28 17:25:28.685792 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:28.685699    2561 generic.go:358] "Generic (PLEG): container finished" podID="31c9a1ab-eb91-4e29-b1eb-c81d58821f46" containerID="523f43306ad4481ffd30fdaa4c481ea390c6edf209ea997a1b21bbe5e6d4e420" exitCode=0
May 28 17:25:28.686233 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:28.685826    2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mtgb7"
May 28 17:25:28.686233 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:28.685840    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mtgb7" event={"ID":"31c9a1ab-eb91-4e29-b1eb-c81d58821f46","Type":"ContainerDied","Data":"523f43306ad4481ffd30fdaa4c481ea390c6edf209ea997a1b21bbe5e6d4e420"}
May 28 17:25:28.686233 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:28.685873    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mtgb7" event={"ID":"31c9a1ab-eb91-4e29-b1eb-c81d58821f46","Type":"ContainerDied","Data":"dde5f447f5eb4e6e3622217d11456761a40e94afad8ba3e5c4b7b834cff59d33"}
May 28 17:25:28.686233 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:28.685888    2561 scope.go:117] "RemoveContainer" containerID="523f43306ad4481ffd30fdaa4c481ea390c6edf209ea997a1b21bbe5e6d4e420"
May 28 17:25:28.694180 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:28.694159    2561 scope.go:117] "RemoveContainer" containerID="523f43306ad4481ffd30fdaa4c481ea390c6edf209ea997a1b21bbe5e6d4e420"
May 28 17:25:28.694462 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:25:28.694443    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"523f43306ad4481ffd30fdaa4c481ea390c6edf209ea997a1b21bbe5e6d4e420\": container with ID starting with 523f43306ad4481ffd30fdaa4c481ea390c6edf209ea997a1b21bbe5e6d4e420 not found: ID does not exist" containerID="523f43306ad4481ffd30fdaa4c481ea390c6edf209ea997a1b21bbe5e6d4e420"
May 28 17:25:28.694504 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:28.694470    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"523f43306ad4481ffd30fdaa4c481ea390c6edf209ea997a1b21bbe5e6d4e420"} err="failed to get container status \"523f43306ad4481ffd30fdaa4c481ea390c6edf209ea997a1b21bbe5e6d4e420\": rpc error: code = NotFound desc = could not find container \"523f43306ad4481ffd30fdaa4c481ea390c6edf209ea997a1b21bbe5e6d4e420\": container with ID starting with 523f43306ad4481ffd30fdaa4c481ea390c6edf209ea997a1b21bbe5e6d4e420 not found: ID does not exist"
May 28 17:25:28.703558 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:28.703533    2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mtgb7"]
May 28 17:25:28.709550 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:28.709528    2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mtgb7"]
May 28 17:25:30.321571 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:30.321528    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31c9a1ab-eb91-4e29-b1eb-c81d58821f46" path="/var/lib/kubelet/pods/31c9a1ab-eb91-4e29-b1eb-c81d58821f46/volumes"
May 28 17:25:35.353153 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:35.353070    2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-d52wt"
May 28 17:25:35.353153 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:35.353114    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-d52wt"
May 28 17:25:35.376897 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:35.376868    2561 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-d52wt"
May 28 17:25:35.729814 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:25:35.729735    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-d52wt"
May 28 17:26:05.427239 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:05.427152    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-wlmfh"]
May 28 17:26:05.427709 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:05.427461    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31c9a1ab-eb91-4e29-b1eb-c81d58821f46" containerName="registry-server"
May 28 17:26:05.427709 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:05.427476    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="31c9a1ab-eb91-4e29-b1eb-c81d58821f46" containerName="registry-server"
May 28 17:26:05.427709 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:05.427544    2561 memory_manager.go:356] "RemoveStaleState removing state" podUID="31c9a1ab-eb91-4e29-b1eb-c81d58821f46" containerName="registry-server"
May 28 17:26:05.430689 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:05.430662    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-wlmfh"
May 28 17:26:05.433648 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:05.433629    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-58bx4\""
May 28 17:26:05.438927 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:05.438906    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-wlmfh"]
May 28 17:26:05.532144 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:05.532103    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4hsg\" (UniqueName: \"kubernetes.io/projected/d8d8d860-4b93-4af2-aec7-dfff155ae059-kube-api-access-g4hsg\") pod \"authorino-operator-657f44b778-wlmfh\" (UID: \"d8d8d860-4b93-4af2-aec7-dfff155ae059\") " pod="kuadrant-system/authorino-operator-657f44b778-wlmfh"
May 28 17:26:05.632863 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:05.632819    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g4hsg\" (UniqueName: \"kubernetes.io/projected/d8d8d860-4b93-4af2-aec7-dfff155ae059-kube-api-access-g4hsg\") pod \"authorino-operator-657f44b778-wlmfh\" (UID: \"d8d8d860-4b93-4af2-aec7-dfff155ae059\") " pod="kuadrant-system/authorino-operator-657f44b778-wlmfh"
May 28 17:26:05.643437 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:05.643405    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4hsg\" (UniqueName: \"kubernetes.io/projected/d8d8d860-4b93-4af2-aec7-dfff155ae059-kube-api-access-g4hsg\") pod \"authorino-operator-657f44b778-wlmfh\" (UID: \"d8d8d860-4b93-4af2-aec7-dfff155ae059\") " pod="kuadrant-system/authorino-operator-657f44b778-wlmfh"
May 28 17:26:05.741149 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:05.741073    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-wlmfh"
May 28 17:26:05.881437 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:05.881406    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-wlmfh"]
May 28 17:26:05.884736 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:26:05.884707    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8d8d860_4b93_4af2_aec7_dfff155ae059.slice/crio-66a92d1e1b1adc6ee03b1b504ea569807636fb3b68c801ff4cf8c1b250ac38c7 WatchSource:0}: Error finding container 66a92d1e1b1adc6ee03b1b504ea569807636fb3b68c801ff4cf8c1b250ac38c7: Status 404 returned error can't find the container with id 66a92d1e1b1adc6ee03b1b504ea569807636fb3b68c801ff4cf8c1b250ac38c7
May 28 17:26:06.805266 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:06.805227    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-wlmfh" event={"ID":"d8d8d860-4b93-4af2-aec7-dfff155ae059","Type":"ContainerStarted","Data":"66a92d1e1b1adc6ee03b1b504ea569807636fb3b68c801ff4cf8c1b250ac38c7"}
May 28 17:26:07.809653 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:07.809614    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-wlmfh" event={"ID":"d8d8d860-4b93-4af2-aec7-dfff155ae059","Type":"ContainerStarted","Data":"2902323607e727b98c0f9f038b8562626e4ace7401a997696c29f08b56ed0106"}
May 28 17:26:07.810084 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:07.809776    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-wlmfh"
May 28 17:26:07.827319 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:07.827254    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-wlmfh" podStartSLOduration=1.520177562 podStartE2EDuration="2.827239628s" podCreationTimestamp="2026-05-28 17:26:05 +0000 UTC" firstStartedPulling="2026-05-28 17:26:05.887187679 +0000 UTC m=+530.126738048" lastFinishedPulling="2026-05-28 17:26:07.194249742 +0000 UTC m=+531.433800114" observedRunningTime="2026-05-28 17:26:07.826154148 +0000 UTC m=+532.065704576" watchObservedRunningTime="2026-05-28 17:26:07.827239628 +0000 UTC m=+532.066790017"
May 28 17:26:18.815553 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:18.815511    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-wlmfh"
May 28 17:26:19.114298 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.114210    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl"]
May 28 17:26:19.118585 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.118567    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl"
May 28 17:26:19.121882 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.121862    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4ckss\""
May 28 17:26:19.139192 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.139170    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl"]
May 28 17:26:19.245961 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.245929    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhh4j\" (UniqueName: \"kubernetes.io/projected/291d1893-d9b3-41d1-bdde-a78007fcafcd-kube-api-access-jhh4j\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-sldpl\" (UID: \"291d1893-d9b3-41d1-bdde-a78007fcafcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl"
May 28 17:26:19.245961 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.245968    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/291d1893-d9b3-41d1-bdde-a78007fcafcd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-sldpl\" (UID: \"291d1893-d9b3-41d1-bdde-a78007fcafcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl"
May 28 17:26:19.346879 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.346842    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jhh4j\" (UniqueName: \"kubernetes.io/projected/291d1893-d9b3-41d1-bdde-a78007fcafcd-kube-api-access-jhh4j\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-sldpl\" (UID: \"291d1893-d9b3-41d1-bdde-a78007fcafcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl"
May 28 17:26:19.347055 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.346895    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/291d1893-d9b3-41d1-bdde-a78007fcafcd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-sldpl\" (UID: \"291d1893-d9b3-41d1-bdde-a78007fcafcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl"
May 28 17:26:19.347366 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.347343    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/291d1893-d9b3-41d1-bdde-a78007fcafcd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-sldpl\" (UID: \"291d1893-d9b3-41d1-bdde-a78007fcafcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl"
May 28 17:26:19.363310 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.363277    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhh4j\" (UniqueName: \"kubernetes.io/projected/291d1893-d9b3-41d1-bdde-a78007fcafcd-kube-api-access-jhh4j\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-sldpl\" (UID: \"291d1893-d9b3-41d1-bdde-a78007fcafcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl"
May 28 17:26:19.387843 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.387777    2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl"]
May 28 17:26:19.388028 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.388016    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl"
May 28 17:26:19.531976 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.531952    2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl"]
May 28 17:26:19.534571 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:26:19.534543    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod291d1893_d9b3_41d1_bdde_a78007fcafcd.slice/crio-22d2ed4f4b982649ff455cc31f0a3275889b037526f772c25a192aaa5849aa01 WatchSource:0}: Error finding container 22d2ed4f4b982649ff455cc31f0a3275889b037526f772c25a192aaa5849aa01: Status 404 returned error can't find the container with id 22d2ed4f4b982649ff455cc31f0a3275889b037526f772c25a192aaa5849aa01
May 28 17:26:19.647196 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.647101    2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl"]
May 28 17:26:19.657796 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.657766    2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl"]
May 28 17:26:19.704730 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.704700    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"]
May 28 17:26:19.709139 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.709119    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"
May 28 17:26:19.718950 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.718926    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"]
May 28 17:26:19.750665 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.750635    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfqcq\" (UniqueName: \"kubernetes.io/projected/5a69a9cf-5c20-4904-b70e-d2f8e1017e0f-kube-api-access-pfqcq\") pod \"kuadrant-operator-controller-manager-84b657d985-89gcw\" (UID: \"5a69a9cf-5c20-4904-b70e-d2f8e1017e0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"
May 28 17:26:19.751016 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.750694    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a69a9cf-5c20-4904-b70e-d2f8e1017e0f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-89gcw\" (UID: \"5a69a9cf-5c20-4904-b70e-d2f8e1017e0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"
May 28 17:26:19.851485 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.851453    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pfqcq\" (UniqueName: \"kubernetes.io/projected/5a69a9cf-5c20-4904-b70e-d2f8e1017e0f-kube-api-access-pfqcq\") pod \"kuadrant-operator-controller-manager-84b657d985-89gcw\" (UID: \"5a69a9cf-5c20-4904-b70e-d2f8e1017e0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"
May 28 17:26:19.851937 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.851492    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a69a9cf-5c20-4904-b70e-d2f8e1017e0f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-89gcw\" (UID: \"5a69a9cf-5c20-4904-b70e-d2f8e1017e0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"
May 28 17:26:19.851937 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.851820    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a69a9cf-5c20-4904-b70e-d2f8e1017e0f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-89gcw\" (UID: \"5a69a9cf-5c20-4904-b70e-d2f8e1017e0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"
May 28 17:26:19.867344 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:19.867320    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfqcq\" (UniqueName: \"kubernetes.io/projected/5a69a9cf-5c20-4904-b70e-d2f8e1017e0f-kube-api-access-pfqcq\") pod \"kuadrant-operator-controller-manager-84b657d985-89gcw\" (UID: \"5a69a9cf-5c20-4904-b70e-d2f8e1017e0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"
May 28 17:26:20.020685 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:20.020589    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"
May 28 17:26:20.181164 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:20.181124    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"]
May 28 17:26:20.184198 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:26:20.184168    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a69a9cf_5c20_4904_b70e_d2f8e1017e0f.slice/crio-0c04e6cd1c87e12266e1764d2e76f2e45426866378a0ecc15caebfc539836774 WatchSource:0}: Error finding container 0c04e6cd1c87e12266e1764d2e76f2e45426866378a0ecc15caebfc539836774: Status 404 returned error can't find the container with id 0c04e6cd1c87e12266e1764d2e76f2e45426866378a0ecc15caebfc539836774
May 28 17:26:20.855089 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:20.855045    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw" event={"ID":"5a69a9cf-5c20-4904-b70e-d2f8e1017e0f","Type":"ContainerStarted","Data":"0c04e6cd1c87e12266e1764d2e76f2e45426866378a0ecc15caebfc539836774"}
May 28 17:26:23.866946 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:23.866901    2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl" podUID="291d1893-d9b3-41d1-bdde-a78007fcafcd" containerName="manager" containerID="cri-o://321b65e9c82f242972401408485341ee9fbd49d4f53b03b60a724e495f2b6c41" gracePeriod=2
May 28 17:26:23.868301 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:23.868278    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw" event={"ID":"5a69a9cf-5c20-4904-b70e-d2f8e1017e0f","Type":"ContainerStarted","Data":"f5ca807ba60afda33c1369206450c8a8e8e8dcb147f7bee21ffad1327016e344"}
May 28 17:26:23.868428 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:23.868417    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"
May 28 17:26:23.891167 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:23.891119    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw" podStartSLOduration=1.693891569 podStartE2EDuration="4.891104294s" podCreationTimestamp="2026-05-28 17:26:19 +0000 UTC" firstStartedPulling="2026-05-28 17:26:20.187044501 +0000 UTC m=+544.426594869" lastFinishedPulling="2026-05-28 17:26:23.384257226 +0000 UTC m=+547.623807594" observedRunningTime="2026-05-28 17:26:23.888789871 +0000 UTC m=+548.128340291" watchObservedRunningTime="2026-05-28 17:26:23.891104294 +0000 UTC m=+548.130654688"
May 28 17:26:24.108302 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:24.108278    2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl"
May 28 17:26:24.110718 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:24.110649    2561 status_manager.go:895] "Failed to get status for pod" podUID="291d1893-d9b3-41d1-bdde-a78007fcafcd" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-sldpl\" is forbidden: User \"system:node:ip-10-0-139-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-178.ec2.internal' and this object"
May 28 17:26:24.189853 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:24.189761    2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhh4j\" (UniqueName: \"kubernetes.io/projected/291d1893-d9b3-41d1-bdde-a78007fcafcd-kube-api-access-jhh4j\") pod \"291d1893-d9b3-41d1-bdde-a78007fcafcd\" (UID: \"291d1893-d9b3-41d1-bdde-a78007fcafcd\") "
May 28 17:26:24.189992 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:24.189857    2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/291d1893-d9b3-41d1-bdde-a78007fcafcd-extensions-socket-volume\") pod \"291d1893-d9b3-41d1-bdde-a78007fcafcd\" (UID: \"291d1893-d9b3-41d1-bdde-a78007fcafcd\") "
May 28 17:26:24.190132 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:24.190107    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/291d1893-d9b3-41d1-bdde-a78007fcafcd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "291d1893-d9b3-41d1-bdde-a78007fcafcd" (UID: "291d1893-d9b3-41d1-bdde-a78007fcafcd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 17:26:24.192016 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:24.191991    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/291d1893-d9b3-41d1-bdde-a78007fcafcd-kube-api-access-jhh4j" (OuterVolumeSpecName: "kube-api-access-jhh4j") pod "291d1893-d9b3-41d1-bdde-a78007fcafcd" (UID: "291d1893-d9b3-41d1-bdde-a78007fcafcd"). InnerVolumeSpecName "kube-api-access-jhh4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:26:24.291286 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:24.291246    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhh4j\" (UniqueName: \"kubernetes.io/projected/291d1893-d9b3-41d1-bdde-a78007fcafcd-kube-api-access-jhh4j\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\""
May 28 17:26:24.291286 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:24.291276    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/291d1893-d9b3-41d1-bdde-a78007fcafcd-extensions-socket-volume\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\""
May 28 17:26:24.321999 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:24.321967    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="291d1893-d9b3-41d1-bdde-a78007fcafcd" path="/var/lib/kubelet/pods/291d1893-d9b3-41d1-bdde-a78007fcafcd/volumes"
May 28 17:26:24.872406 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:24.872372    2561 generic.go:358] "Generic (PLEG): container finished" podID="291d1893-d9b3-41d1-bdde-a78007fcafcd" containerID="321b65e9c82f242972401408485341ee9fbd49d4f53b03b60a724e495f2b6c41" exitCode=2
May 28 17:26:24.872883 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:24.872419    2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl"
May 28 17:26:24.872883 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:24.872446    2561 scope.go:117] "RemoveContainer" containerID="321b65e9c82f242972401408485341ee9fbd49d4f53b03b60a724e495f2b6c41"
May 28 17:26:24.877311 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:24.877283    2561 status_manager.go:895] "Failed to get status for pod" podUID="291d1893-d9b3-41d1-bdde-a78007fcafcd" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-sldpl\" is forbidden: User \"system:node:ip-10-0-139-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-178.ec2.internal' and this object"
May 28 17:26:24.880245 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:24.880228    2561 scope.go:117] "RemoveContainer" containerID="321b65e9c82f242972401408485341ee9fbd49d4f53b03b60a724e495f2b6c41"
May 28 17:26:24.880481 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:26:24.880464    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"321b65e9c82f242972401408485341ee9fbd49d4f53b03b60a724e495f2b6c41\": container with ID starting with 321b65e9c82f242972401408485341ee9fbd49d4f53b03b60a724e495f2b6c41 not found: ID does not exist" containerID="321b65e9c82f242972401408485341ee9fbd49d4f53b03b60a724e495f2b6c41"
May 28 17:26:24.880536 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:24.880489    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"321b65e9c82f242972401408485341ee9fbd49d4f53b03b60a724e495f2b6c41"} err="failed to get container status \"321b65e9c82f242972401408485341ee9fbd49d4f53b03b60a724e495f2b6c41\": rpc error: code = NotFound desc = could not find container \"321b65e9c82f242972401408485341ee9fbd49d4f53b03b60a724e495f2b6c41\": container with ID starting with 321b65e9c82f242972401408485341ee9fbd49d4f53b03b60a724e495f2b6c41 not found: ID does not exist"
May 28 17:26:26.321268 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:26.321230    2561 status_manager.go:895] "Failed to get status for pod" podUID="291d1893-d9b3-41d1-bdde-a78007fcafcd" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-sldpl" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-sldpl\" is forbidden: User \"system:node:ip-10-0-139-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-178.ec2.internal' and this object"
May 28 17:26:34.875346 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:34.875309    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"
May 28 17:26:47.127085 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.127040    2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"]
May 28 17:26:47.127746 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.127661    2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw" podUID="5a69a9cf-5c20-4904-b70e-d2f8e1017e0f" containerName="manager" containerID="cri-o://f5ca807ba60afda33c1369206450c8a8e8e8dcb147f7bee21ffad1327016e344" gracePeriod=10
May 28 17:26:47.396211 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.396182    2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"
May 28 17:26:47.492238 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.492201    2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a69a9cf-5c20-4904-b70e-d2f8e1017e0f-extensions-socket-volume\") pod \"5a69a9cf-5c20-4904-b70e-d2f8e1017e0f\" (UID: \"5a69a9cf-5c20-4904-b70e-d2f8e1017e0f\") "
May 28 17:26:47.492427 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.492269    2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfqcq\" (UniqueName: \"kubernetes.io/projected/5a69a9cf-5c20-4904-b70e-d2f8e1017e0f-kube-api-access-pfqcq\") pod \"5a69a9cf-5c20-4904-b70e-d2f8e1017e0f\" (UID: \"5a69a9cf-5c20-4904-b70e-d2f8e1017e0f\") "
May 28 17:26:47.492760 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.492732    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a69a9cf-5c20-4904-b70e-d2f8e1017e0f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5a69a9cf-5c20-4904-b70e-d2f8e1017e0f" (UID: "5a69a9cf-5c20-4904-b70e-d2f8e1017e0f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 17:26:47.494607 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.494580    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a69a9cf-5c20-4904-b70e-d2f8e1017e0f-kube-api-access-pfqcq" (OuterVolumeSpecName: "kube-api-access-pfqcq") pod "5a69a9cf-5c20-4904-b70e-d2f8e1017e0f" (UID: "5a69a9cf-5c20-4904-b70e-d2f8e1017e0f"). InnerVolumeSpecName "kube-api-access-pfqcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:26:47.593436 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.593403    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a69a9cf-5c20-4904-b70e-d2f8e1017e0f-extensions-socket-volume\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\""
May 28 17:26:47.593436 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.593433    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfqcq\" (UniqueName: \"kubernetes.io/projected/5a69a9cf-5c20-4904-b70e-d2f8e1017e0f-kube-api-access-pfqcq\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\""
May 28 17:26:47.948600 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.948560    2561 generic.go:358] "Generic (PLEG): container finished" podID="5a69a9cf-5c20-4904-b70e-d2f8e1017e0f" containerID="f5ca807ba60afda33c1369206450c8a8e8e8dcb147f7bee21ffad1327016e344" exitCode=0
May 28 17:26:47.948600 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.948604    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw" event={"ID":"5a69a9cf-5c20-4904-b70e-d2f8e1017e0f","Type":"ContainerDied","Data":"f5ca807ba60afda33c1369206450c8a8e8e8dcb147f7bee21ffad1327016e344"}
May 28 17:26:47.948882 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.948630    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw" event={"ID":"5a69a9cf-5c20-4904-b70e-d2f8e1017e0f","Type":"ContainerDied","Data":"0c04e6cd1c87e12266e1764d2e76f2e45426866378a0ecc15caebfc539836774"}
May 28 17:26:47.948882 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.948639    2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"
May 28 17:26:47.948882 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.948654    2561 scope.go:117] "RemoveContainer" containerID="f5ca807ba60afda33c1369206450c8a8e8e8dcb147f7bee21ffad1327016e344"
May 28 17:26:47.956856 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.956835    2561 scope.go:117] "RemoveContainer" containerID="f5ca807ba60afda33c1369206450c8a8e8e8dcb147f7bee21ffad1327016e344"
May 28 17:26:47.957121 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:26:47.957102    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5ca807ba60afda33c1369206450c8a8e8e8dcb147f7bee21ffad1327016e344\": container with ID starting with f5ca807ba60afda33c1369206450c8a8e8e8dcb147f7bee21ffad1327016e344 not found: ID does not exist" containerID="f5ca807ba60afda33c1369206450c8a8e8e8dcb147f7bee21ffad1327016e344"
May 28 17:26:47.957168 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.957129    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5ca807ba60afda33c1369206450c8a8e8e8dcb147f7bee21ffad1327016e344"} err="failed to get container status \"f5ca807ba60afda33c1369206450c8a8e8e8dcb147f7bee21ffad1327016e344\": rpc error: code = NotFound desc = could not find container \"f5ca807ba60afda33c1369206450c8a8e8e8dcb147f7bee21ffad1327016e344\": container with ID starting with f5ca807ba60afda33c1369206450c8a8e8e8dcb147f7bee21ffad1327016e344 not found: ID does not exist"
May 28 17:26:47.974296 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.974262    2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"]
May 28 17:26:47.978706 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:47.978664    2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-89gcw"]
May 28 17:26:48.321382 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:26:48.321345    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a69a9cf-5c20-4904-b70e-d2f8e1017e0f" path="/var/lib/kubelet/pods/5a69a9cf-5c20-4904-b70e-d2f8e1017e0f/volumes"
May 28 17:27:06.965968 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:06.965932    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-95m2k"]
May 28 17:27:06.966442 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:06.966215    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a69a9cf-5c20-4904-b70e-d2f8e1017e0f" containerName="manager"
May 28 17:27:06.966442 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:06.966227    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a69a9cf-5c20-4904-b70e-d2f8e1017e0f" containerName="manager"
May 28 17:27:06.966442 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:06.966243    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="291d1893-d9b3-41d1-bdde-a78007fcafcd" containerName="manager"
May 28 17:27:06.966442 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:06.966251    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="291d1893-d9b3-41d1-bdde-a78007fcafcd" containerName="manager"
May 28 17:27:06.966442 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:06.966305    2561 memory_manager.go:356] "RemoveStaleState removing state" podUID="291d1893-d9b3-41d1-bdde-a78007fcafcd" containerName="manager"
May 28 17:27:06.966442 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:06.966315    2561 memory_manager.go:356] "RemoveStaleState removing state" podUID="5a69a9cf-5c20-4904-b70e-d2f8e1017e0f" containerName="manager"
May 28 17:27:06.968085 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:06.968070    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k"
May 28 17:27:06.971028 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:06.971003    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-nd8c6\""
May 28 17:27:06.971132 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:06.971006    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
May 28 17:27:06.977028 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:06.977004    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-95m2k"]
May 28 17:27:07.036731 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:07.036691    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlcbs\" (UniqueName: \"kubernetes.io/projected/af1dec41-828b-44b4-9c56-a20179ad12bc-kube-api-access-mlcbs\") pod \"limitador-limitador-7d549b5b-95m2k\" (UID: \"af1dec41-828b-44b4-9c56-a20179ad12bc\") " pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k"
May 28 17:27:07.036902 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:07.036865    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/af1dec41-828b-44b4-9c56-a20179ad12bc-config-file\") pod \"limitador-limitador-7d549b5b-95m2k\" (UID: \"af1dec41-828b-44b4-9c56-a20179ad12bc\") " pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k"
May 28 17:27:07.066108 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:07.066069    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-95m2k"]
May 28 17:27:07.137518 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:07.137484    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/af1dec41-828b-44b4-9c56-a20179ad12bc-config-file\") pod \"limitador-limitador-7d549b5b-95m2k\" (UID: \"af1dec41-828b-44b4-9c56-a20179ad12bc\") " pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k"
May 28 17:27:07.137721 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:07.137528    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mlcbs\" (UniqueName: \"kubernetes.io/projected/af1dec41-828b-44b4-9c56-a20179ad12bc-kube-api-access-mlcbs\") pod \"limitador-limitador-7d549b5b-95m2k\" (UID: \"af1dec41-828b-44b4-9c56-a20179ad12bc\") " pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k"
May 28 17:27:07.138120 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:07.138103    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/af1dec41-828b-44b4-9c56-a20179ad12bc-config-file\") pod \"limitador-limitador-7d549b5b-95m2k\" (UID: \"af1dec41-828b-44b4-9c56-a20179ad12bc\") " pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k"
May 28 17:27:07.146449 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:07.146427    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlcbs\" (UniqueName: \"kubernetes.io/projected/af1dec41-828b-44b4-9c56-a20179ad12bc-kube-api-access-mlcbs\") pod \"limitador-limitador-7d549b5b-95m2k\" (UID: \"af1dec41-828b-44b4-9c56-a20179ad12bc\") " pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k"
May 28 17:27:07.279745 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:07.279645    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k"
May 28 17:27:07.447710 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:07.444330    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-95m2k"]
May 28 17:27:07.451031 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:27:07.451011    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf1dec41_828b_44b4_9c56_a20179ad12bc.slice/crio-70070607ff5e06ac5d04e3fac3bd08ca44b58c919592632f48a333ccf188944f WatchSource:0}: Error finding container 70070607ff5e06ac5d04e3fac3bd08ca44b58c919592632f48a333ccf188944f: Status 404 returned error can't find the container with id 70070607ff5e06ac5d04e3fac3bd08ca44b58c919592632f48a333ccf188944f
May 28 17:27:08.026244 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:08.026180    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k" event={"ID":"af1dec41-828b-44b4-9c56-a20179ad12bc","Type":"ContainerStarted","Data":"70070607ff5e06ac5d04e3fac3bd08ca44b58c919592632f48a333ccf188944f"}
May 28 17:27:11.039868 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:11.039830    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k" event={"ID":"af1dec41-828b-44b4-9c56-a20179ad12bc","Type":"ContainerStarted","Data":"bb1fc6f08f00799fd63f1e54ff2f0a21e5c0554238cdd21e374155fb71cbac07"}
May 28 17:27:11.040319 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:11.039960    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k"
May 28 17:27:11.060396 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:11.060343    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k" podStartSLOduration=2.532181853 podStartE2EDuration="5.06032738s" podCreationTimestamp="2026-05-28 17:27:06 +0000 UTC" firstStartedPulling="2026-05-28 17:27:07.452591743 +0000 UTC m=+591.692142111" lastFinishedPulling="2026-05-28 17:27:09.980737271 +0000 UTC m=+594.220287638" observedRunningTime="2026-05-28 17:27:11.058977743 +0000 UTC m=+595.298528137" watchObservedRunningTime="2026-05-28 17:27:11.06032738 +0000 UTC m=+595.299877769"
May 28 17:27:22.044747 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:22.044709    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k"
May 28 17:27:22.268218 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:22.268175    2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-95m2k"]
May 28 17:27:22.268446 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:22.268421    2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k" podUID="af1dec41-828b-44b4-9c56-a20179ad12bc" containerName="limitador" containerID="cri-o://bb1fc6f08f00799fd63f1e54ff2f0a21e5c0554238cdd21e374155fb71cbac07" gracePeriod=30
May 28 17:27:22.813524 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:22.813503    2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k"
May 28 17:27:22.864820 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:22.864745    2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/af1dec41-828b-44b4-9c56-a20179ad12bc-config-file\") pod \"af1dec41-828b-44b4-9c56-a20179ad12bc\" (UID: \"af1dec41-828b-44b4-9c56-a20179ad12bc\") "
May 28 17:27:22.864820 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:22.864808    2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlcbs\" (UniqueName: \"kubernetes.io/projected/af1dec41-828b-44b4-9c56-a20179ad12bc-kube-api-access-mlcbs\") pod \"af1dec41-828b-44b4-9c56-a20179ad12bc\" (UID: \"af1dec41-828b-44b4-9c56-a20179ad12bc\") "
May 28 17:27:22.865095 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:22.865073    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af1dec41-828b-44b4-9c56-a20179ad12bc-config-file" (OuterVolumeSpecName: "config-file") pod "af1dec41-828b-44b4-9c56-a20179ad12bc" (UID: "af1dec41-828b-44b4-9c56-a20179ad12bc"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 17:27:22.866964 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:22.866933    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af1dec41-828b-44b4-9c56-a20179ad12bc-kube-api-access-mlcbs" (OuterVolumeSpecName: "kube-api-access-mlcbs") pod "af1dec41-828b-44b4-9c56-a20179ad12bc" (UID: "af1dec41-828b-44b4-9c56-a20179ad12bc"). InnerVolumeSpecName "kube-api-access-mlcbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:27:22.966305 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:22.966265    2561 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/af1dec41-828b-44b4-9c56-a20179ad12bc-config-file\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\""
May 28 17:27:22.966305 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:22.966301    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlcbs\" (UniqueName: \"kubernetes.io/projected/af1dec41-828b-44b4-9c56-a20179ad12bc-kube-api-access-mlcbs\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\""
May 28 17:27:23.077857 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:23.077819    2561 generic.go:358] "Generic (PLEG): container finished" podID="af1dec41-828b-44b4-9c56-a20179ad12bc" containerID="bb1fc6f08f00799fd63f1e54ff2f0a21e5c0554238cdd21e374155fb71cbac07" exitCode=0
May 28 17:27:23.078237 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:23.077882    2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k"
May 28 17:27:23.078237 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:23.077907    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k" event={"ID":"af1dec41-828b-44b4-9c56-a20179ad12bc","Type":"ContainerDied","Data":"bb1fc6f08f00799fd63f1e54ff2f0a21e5c0554238cdd21e374155fb71cbac07"}
May 28 17:27:23.078237 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:23.077946    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-95m2k" event={"ID":"af1dec41-828b-44b4-9c56-a20179ad12bc","Type":"ContainerDied","Data":"70070607ff5e06ac5d04e3fac3bd08ca44b58c919592632f48a333ccf188944f"}
May 28 17:27:23.078237 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:23.077964    2561 scope.go:117] "RemoveContainer" containerID="bb1fc6f08f00799fd63f1e54ff2f0a21e5c0554238cdd21e374155fb71cbac07"
May 28 17:27:23.086522 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:23.086501    2561 scope.go:117] "RemoveContainer" containerID="bb1fc6f08f00799fd63f1e54ff2f0a21e5c0554238cdd21e374155fb71cbac07"
May 28 17:27:23.086805 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:27:23.086779    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb1fc6f08f00799fd63f1e54ff2f0a21e5c0554238cdd21e374155fb71cbac07\": container with ID starting with bb1fc6f08f00799fd63f1e54ff2f0a21e5c0554238cdd21e374155fb71cbac07 not found: ID does not exist" containerID="bb1fc6f08f00799fd63f1e54ff2f0a21e5c0554238cdd21e374155fb71cbac07"
May 28 17:27:23.086884 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:23.086813    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb1fc6f08f00799fd63f1e54ff2f0a21e5c0554238cdd21e374155fb71cbac07"} err="failed to get container status \"bb1fc6f08f00799fd63f1e54ff2f0a21e5c0554238cdd21e374155fb71cbac07\": rpc error: code = NotFound desc = could not find container \"bb1fc6f08f00799fd63f1e54ff2f0a21e5c0554238cdd21e374155fb71cbac07\": container with ID starting with bb1fc6f08f00799fd63f1e54ff2f0a21e5c0554238cdd21e374155fb71cbac07 not found: ID does not exist"
May 28 17:27:23.099514 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:23.099490    2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-95m2k"]
May 28 17:27:23.107480 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:23.104585    2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-95m2k"]
May 28 17:27:24.321447 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:24.321412    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af1dec41-828b-44b4-9c56-a20179ad12bc" path="/var/lib/kubelet/pods/af1dec41-828b-44b4-9c56-a20179ad12bc/volumes"
May 28 17:27:26.431332 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:26.431299    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-bwnfb"]
May 28 17:27:26.431703 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:26.431580    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af1dec41-828b-44b4-9c56-a20179ad12bc" containerName="limitador"
May 28 17:27:26.431703 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:26.431590    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="af1dec41-828b-44b4-9c56-a20179ad12bc" containerName="limitador"
May 28 17:27:26.431703 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:26.431646    2561 memory_manager.go:356] "RemoveStaleState removing state" podUID="af1dec41-828b-44b4-9c56-a20179ad12bc" containerName="limitador"
May 28 17:27:26.440449 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:26.439750    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-bwnfb"
May 28 17:27:26.441641 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:26.441609    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-bwnfb"]
May 28 17:27:26.442762 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:26.442741    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
May 28 17:27:26.442880 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:26.442762    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-bf9q9\""
May 28 17:27:26.501000 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:26.500962    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/643025d4-e9cf-4dd2-a876-673ddf5fc2db-data\") pod \"postgres-68c4fbbd6f-bwnfb\" (UID: \"643025d4-e9cf-4dd2-a876-673ddf5fc2db\") " pod="opendatahub/postgres-68c4fbbd6f-bwnfb"
May 28 17:27:26.501180 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:26.501016    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qzpv\" (UniqueName: \"kubernetes.io/projected/643025d4-e9cf-4dd2-a876-673ddf5fc2db-kube-api-access-8qzpv\") pod \"postgres-68c4fbbd6f-bwnfb\" (UID: \"643025d4-e9cf-4dd2-a876-673ddf5fc2db\") " pod="opendatahub/postgres-68c4fbbd6f-bwnfb"
May 28 17:27:26.602271 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:26.602232    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/643025d4-e9cf-4dd2-a876-673ddf5fc2db-data\") pod \"postgres-68c4fbbd6f-bwnfb\" (UID: \"643025d4-e9cf-4dd2-a876-673ddf5fc2db\") " pod="opendatahub/postgres-68c4fbbd6f-bwnfb"
May 28 17:27:26.602441 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:26.602283    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8qzpv\" (UniqueName: \"kubernetes.io/projected/643025d4-e9cf-4dd2-a876-673ddf5fc2db-kube-api-access-8qzpv\") pod \"postgres-68c4fbbd6f-bwnfb\" (UID: \"643025d4-e9cf-4dd2-a876-673ddf5fc2db\") " pod="opendatahub/postgres-68c4fbbd6f-bwnfb"
May 28 17:27:26.602619 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:26.602600    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/643025d4-e9cf-4dd2-a876-673ddf5fc2db-data\") pod \"postgres-68c4fbbd6f-bwnfb\" (UID: \"643025d4-e9cf-4dd2-a876-673ddf5fc2db\") " pod="opendatahub/postgres-68c4fbbd6f-bwnfb"
May 28 17:27:26.611808 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:26.611776    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qzpv\" (UniqueName: \"kubernetes.io/projected/643025d4-e9cf-4dd2-a876-673ddf5fc2db-kube-api-access-8qzpv\") pod \"postgres-68c4fbbd6f-bwnfb\" (UID: \"643025d4-e9cf-4dd2-a876-673ddf5fc2db\") " pod="opendatahub/postgres-68c4fbbd6f-bwnfb"
May 28 17:27:26.752100 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:26.752008    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-bwnfb"
May 28 17:27:26.880236 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:26.880204    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-bwnfb"]
May 28 17:27:26.883135 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:27:26.883108    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod643025d4_e9cf_4dd2_a876_673ddf5fc2db.slice/crio-05341f8824513bd8fdfc93a996de69b325282ec92a990f1755e49e23e4a6a9dd WatchSource:0}: Error finding container 05341f8824513bd8fdfc93a996de69b325282ec92a990f1755e49e23e4a6a9dd: Status 404 returned error can't find the container with id 05341f8824513bd8fdfc93a996de69b325282ec92a990f1755e49e23e4a6a9dd
May 28 17:27:27.091987 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:27.091948    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-bwnfb" event={"ID":"643025d4-e9cf-4dd2-a876-673ddf5fc2db","Type":"ContainerStarted","Data":"05341f8824513bd8fdfc93a996de69b325282ec92a990f1755e49e23e4a6a9dd"}
May 28 17:27:32.109757 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:32.109720    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-bwnfb" event={"ID":"643025d4-e9cf-4dd2-a876-673ddf5fc2db","Type":"ContainerStarted","Data":"ebc80dbea638a8cd825d7555a47a79bb74710a0dd245b1c4408e92bbf4dee947"}
May 28 17:27:32.110150 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:32.109845    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-bwnfb"
May 28 17:27:32.129400 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:32.129353    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-bwnfb" podStartSLOduration=1.27950933 podStartE2EDuration="6.129339664s" podCreationTimestamp="2026-05-28 17:27:26 +0000 UTC" firstStartedPulling="2026-05-28 17:27:26.884366404 +0000 UTC m=+611.123916772" lastFinishedPulling="2026-05-28 17:27:31.734196719 +0000 UTC m=+615.973747106" observedRunningTime="2026-05-28 17:27:32.128181506 +0000 UTC m=+616.367731893" watchObservedRunningTime="2026-05-28 17:27:32.129339664 +0000 UTC m=+616.368890103"
May 28 17:27:38.142774 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:38.142743    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-bwnfb"
May 28 17:27:44.041589 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.041553    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6589b49577-z7zvd"]
May 28 17:27:44.063891 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.063860    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6589b49577-z7zvd"]
May 28 17:27:44.064083 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.063977    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6589b49577-z7zvd"
May 28 17:27:44.067032 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.067006    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bfnrd\""
May 28 17:27:44.067175 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.067015    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
May 28 17:27:44.068197 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.068177    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
May 28 17:27:44.155989 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.155955    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146-webhook-cert\") pod \"maas-controller-6589b49577-z7zvd\" (UID: \"d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146\") " pod="opendatahub/maas-controller-6589b49577-z7zvd"
May 28 17:27:44.156172 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.156021    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-554bz\" (UniqueName: \"kubernetes.io/projected/d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146-kube-api-access-554bz\") pod \"maas-controller-6589b49577-z7zvd\" (UID: \"d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146\") " pod="opendatahub/maas-controller-6589b49577-z7zvd"
May 28 17:27:44.205200 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.205168    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-865f658498-4g5z8"]
May 28 17:27:44.207623 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.207606    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865f658498-4g5z8"
May 28 17:27:44.218015 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.217992    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-865f658498-4g5z8"]
May 28 17:27:44.256610 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.256579    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-554bz\" (UniqueName: \"kubernetes.io/projected/d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146-kube-api-access-554bz\") pod \"maas-controller-6589b49577-z7zvd\" (UID: \"d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146\") " pod="opendatahub/maas-controller-6589b49577-z7zvd"
May 28 17:27:44.256787 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.256627    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146-webhook-cert\") pod \"maas-controller-6589b49577-z7zvd\" (UID: \"d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146\") " pod="opendatahub/maas-controller-6589b49577-z7zvd"
May 28 17:27:44.259043 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.259024    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146-webhook-cert\") pod \"maas-controller-6589b49577-z7zvd\" (UID: \"d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146\") " pod="opendatahub/maas-controller-6589b49577-z7zvd"
May 28 17:27:44.265005 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.264983    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-554bz\" (UniqueName: \"kubernetes.io/projected/d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146-kube-api-access-554bz\") pod \"maas-controller-6589b49577-z7zvd\" (UID: \"d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146\") " pod="opendatahub/maas-controller-6589b49577-z7zvd"
May 28 17:27:44.357937 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.357896    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed8ab7e4-40a1-430c-95d3-dd091dcae337-webhook-cert\") pod \"maas-controller-865f658498-4g5z8\" (UID: \"ed8ab7e4-40a1-430c-95d3-dd091dcae337\") " pod="opendatahub/maas-controller-865f658498-4g5z8"
May 28 17:27:44.358120 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.357954    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvfg8\" (UniqueName: \"kubernetes.io/projected/ed8ab7e4-40a1-430c-95d3-dd091dcae337-kube-api-access-rvfg8\") pod \"maas-controller-865f658498-4g5z8\" (UID: \"ed8ab7e4-40a1-430c-95d3-dd091dcae337\") " pod="opendatahub/maas-controller-865f658498-4g5z8"
May 28 17:27:44.376748 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.376721    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6589b49577-z7zvd"
May 28 17:27:44.459598 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.459168    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rvfg8\" (UniqueName: \"kubernetes.io/projected/ed8ab7e4-40a1-430c-95d3-dd091dcae337-kube-api-access-rvfg8\") pod \"maas-controller-865f658498-4g5z8\" (UID: \"ed8ab7e4-40a1-430c-95d3-dd091dcae337\") " pod="opendatahub/maas-controller-865f658498-4g5z8"
May 28 17:27:44.459788 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.459669    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed8ab7e4-40a1-430c-95d3-dd091dcae337-webhook-cert\") pod \"maas-controller-865f658498-4g5z8\" (UID: \"ed8ab7e4-40a1-430c-95d3-dd091dcae337\") " pod="opendatahub/maas-controller-865f658498-4g5z8"
May 28 17:27:44.464339 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.464315    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed8ab7e4-40a1-430c-95d3-dd091dcae337-webhook-cert\") pod \"maas-controller-865f658498-4g5z8\" (UID: \"ed8ab7e4-40a1-430c-95d3-dd091dcae337\") " pod="opendatahub/maas-controller-865f658498-4g5z8"
May 28 17:27:44.468650 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.468628    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvfg8\" (UniqueName: \"kubernetes.io/projected/ed8ab7e4-40a1-430c-95d3-dd091dcae337-kube-api-access-rvfg8\") pod \"maas-controller-865f658498-4g5z8\" (UID: \"ed8ab7e4-40a1-430c-95d3-dd091dcae337\") " pod="opendatahub/maas-controller-865f658498-4g5z8"
May 28 17:27:44.510133 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.510104    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6589b49577-z7zvd"]
May 28 17:27:44.512458 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:27:44.512432    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd20b3ffe_8e8a_4cf9_8ed9_7c58afc0c146.slice/crio-0801681fe9a7453d8454b075dcec480bd27d97a13fea8bccc9b65aa43da79d75 WatchSource:0}: Error finding container 0801681fe9a7453d8454b075dcec480bd27d97a13fea8bccc9b65aa43da79d75: Status 404 returned error can't find the container with id 0801681fe9a7453d8454b075dcec480bd27d97a13fea8bccc9b65aa43da79d75
May 28 17:27:44.513735 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.513717    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 17:27:44.518346 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.518322    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865f658498-4g5z8"
May 28 17:27:44.647591 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:44.647554    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-865f658498-4g5z8"]
May 28 17:27:44.649592 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:27:44.649569    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded8ab7e4_40a1_430c_95d3_dd091dcae337.slice/crio-96c7e365ea7ebcd8800f70b649f63a9f1d52f222129c6c0074c5c2c448a358c1 WatchSource:0}: Error finding container 96c7e365ea7ebcd8800f70b649f63a9f1d52f222129c6c0074c5c2c448a358c1: Status 404 returned error can't find the container with id 96c7e365ea7ebcd8800f70b649f63a9f1d52f222129c6c0074c5c2c448a358c1
May 28 17:27:45.160211 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:45.160172    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865f658498-4g5z8" event={"ID":"ed8ab7e4-40a1-430c-95d3-dd091dcae337","Type":"ContainerStarted","Data":"96c7e365ea7ebcd8800f70b649f63a9f1d52f222129c6c0074c5c2c448a358c1"}
May 28 17:27:45.161786 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:45.161740    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6589b49577-z7zvd" event={"ID":"d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146","Type":"ContainerStarted","Data":"0801681fe9a7453d8454b075dcec480bd27d97a13fea8bccc9b65aa43da79d75"}
May 28 17:27:48.173383 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:48.173347    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865f658498-4g5z8" event={"ID":"ed8ab7e4-40a1-430c-95d3-dd091dcae337","Type":"ContainerStarted","Data":"feca6fbc99ac95feaef28f2b87eb37b748360c2a4538abeec4773a22bc357809"}
May 28 17:27:48.173873 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:48.173582    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-865f658498-4g5z8"
May 28 17:27:48.174731 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:48.174703    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6589b49577-z7zvd" event={"ID":"d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146","Type":"ContainerStarted","Data":"38a7ee127923e49ef7e05fe61889d69ed58e5ce6ea03d14a4fef62d57769ba11"}
May 28 17:27:48.174863 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:48.174849    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6589b49577-z7zvd"
May 28 17:27:48.191414 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:48.191375    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-865f658498-4g5z8" podStartSLOduration=1.6266938309999999 podStartE2EDuration="4.191363585s" podCreationTimestamp="2026-05-28 17:27:44 +0000 UTC" firstStartedPulling="2026-05-28 17:27:44.650841306 +0000 UTC m=+628.890391674" lastFinishedPulling="2026-05-28 17:27:47.215511059 +0000 UTC m=+631.455061428" observedRunningTime="2026-05-28 17:27:48.190234162 +0000 UTC m=+632.429784552" watchObservedRunningTime="2026-05-28 17:27:48.191363585 +0000 UTC m=+632.430913975"
May 28 17:27:48.208760 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:48.208713    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-6589b49577-z7zvd" podStartSLOduration=1.5162505670000002 podStartE2EDuration="4.208698873s" podCreationTimestamp="2026-05-28 17:27:44 +0000 UTC" firstStartedPulling="2026-05-28 17:27:44.513878282 +0000 UTC m=+628.753428649" lastFinishedPulling="2026-05-28 17:27:47.206326584 +0000 UTC m=+631.445876955" observedRunningTime="2026-05-28 17:27:48.207000188 +0000 UTC m=+632.446550579" watchObservedRunningTime="2026-05-28 17:27:48.208698873 +0000 UTC m=+632.448249263"
May 28 17:27:59.183223 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:59.183192    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-865f658498-4g5z8"
May 28 17:27:59.183665 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:59.183347    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6589b49577-z7zvd"
May 28 17:27:59.237294 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:59.237259    2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6589b49577-z7zvd"]
May 28 17:27:59.237554 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:59.237524    2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-6589b49577-z7zvd" podUID="d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146" containerName="manager" containerID="cri-o://38a7ee127923e49ef7e05fe61889d69ed58e5ce6ea03d14a4fef62d57769ba11" gracePeriod=30
May 28 17:27:59.479603 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:59.479577    2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6589b49577-z7zvd"
May 28 17:27:59.487670 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:59.487644    2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-554bz\" (UniqueName: \"kubernetes.io/projected/d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146-kube-api-access-554bz\") pod \"d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146\" (UID: \"d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146\") "
May 28 17:27:59.487835 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:59.487716    2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146-webhook-cert\") pod \"d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146\" (UID: \"d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146\") "
May 28 17:27:59.489919 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:59.489893    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146-kube-api-access-554bz" (OuterVolumeSpecName: "kube-api-access-554bz") pod "d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146" (UID: "d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146"). InnerVolumeSpecName "kube-api-access-554bz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:27:59.490038 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:59.489961    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146" (UID: "d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 17:27:59.588356 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:59.588320    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-554bz\" (UniqueName: \"kubernetes.io/projected/d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146-kube-api-access-554bz\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\""
May 28 17:27:59.588356 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:27:59.588351    2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146-webhook-cert\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\""
May 28 17:28:00.212845 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:00.212809    2561 generic.go:358] "Generic (PLEG): container finished" podID="d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146" containerID="38a7ee127923e49ef7e05fe61889d69ed58e5ce6ea03d14a4fef62d57769ba11" exitCode=0
May 28 17:28:00.213332 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:00.212873    2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6589b49577-z7zvd"
May 28 17:28:00.213332 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:00.212899    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6589b49577-z7zvd" event={"ID":"d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146","Type":"ContainerDied","Data":"38a7ee127923e49ef7e05fe61889d69ed58e5ce6ea03d14a4fef62d57769ba11"}
May 28 17:28:00.213332 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:00.212956    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6589b49577-z7zvd" event={"ID":"d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146","Type":"ContainerDied","Data":"0801681fe9a7453d8454b075dcec480bd27d97a13fea8bccc9b65aa43da79d75"}
May 28 17:28:00.213332 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:00.212977    2561 scope.go:117] "RemoveContainer" containerID="38a7ee127923e49ef7e05fe61889d69ed58e5ce6ea03d14a4fef62d57769ba11"
May 28 17:28:00.222053 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:00.222033    2561 scope.go:117] "RemoveContainer" containerID="38a7ee127923e49ef7e05fe61889d69ed58e5ce6ea03d14a4fef62d57769ba11"
May 28 17:28:00.222330 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:28:00.222311    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38a7ee127923e49ef7e05fe61889d69ed58e5ce6ea03d14a4fef62d57769ba11\": container with ID starting with 38a7ee127923e49ef7e05fe61889d69ed58e5ce6ea03d14a4fef62d57769ba11 not found: ID does not exist" containerID="38a7ee127923e49ef7e05fe61889d69ed58e5ce6ea03d14a4fef62d57769ba11"
May 28 17:28:00.222385 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:00.222347    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a7ee127923e49ef7e05fe61889d69ed58e5ce6ea03d14a4fef62d57769ba11"} err="failed to get container status \"38a7ee127923e49ef7e05fe61889d69ed58e5ce6ea03d14a4fef62d57769ba11\": rpc error: code = NotFound desc = could not find container \"38a7ee127923e49ef7e05fe61889d69ed58e5ce6ea03d14a4fef62d57769ba11\": container with ID starting with 38a7ee127923e49ef7e05fe61889d69ed58e5ce6ea03d14a4fef62d57769ba11 not found: ID does not exist"
May 28 17:28:00.235504 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:00.235478    2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6589b49577-z7zvd"]
May 28 17:28:00.239259 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:00.239236    2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6589b49577-z7zvd"]
May 28 17:28:00.321616 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:00.321585    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146" path="/var/lib/kubelet/pods/d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146/volumes"
May 28 17:28:18.270543 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.270510    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-8fd8f878b-dwvxp"]
May 28 17:28:18.270967 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.270839    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146" containerName="manager"
May 28 17:28:18.270967 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.270851    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146" containerName="manager"
May 28 17:28:18.270967 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.270916    2561 memory_manager.go:356] "RemoveStaleState removing state" podUID="d20b3ffe-8e8a-4cf9-8ed9-7c58afc0c146" containerName="manager"
May 28 17:28:18.273264 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.273244    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-8fd8f878b-dwvxp"
May 28 17:28:18.275929 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.275911    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-nf2kl\""
May 28 17:28:18.276033 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.275952    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
May 28 17:28:18.286860 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.286831    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-8fd8f878b-dwvxp"]
May 28 17:28:18.345064 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.345030    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff0b889f-0e6d-4954-9c3d-e534fe15075c-maas-api-tls\") pod \"maas-api-8fd8f878b-dwvxp\" (UID: \"ff0b889f-0e6d-4954-9c3d-e534fe15075c\") " pod="opendatahub/maas-api-8fd8f878b-dwvxp"
May 28 17:28:18.345226 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.345091    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8clx\" (UniqueName: \"kubernetes.io/projected/ff0b889f-0e6d-4954-9c3d-e534fe15075c-kube-api-access-x8clx\") pod \"maas-api-8fd8f878b-dwvxp\" (UID: \"ff0b889f-0e6d-4954-9c3d-e534fe15075c\") " pod="opendatahub/maas-api-8fd8f878b-dwvxp"
May 28 17:28:18.445746 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.445708    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff0b889f-0e6d-4954-9c3d-e534fe15075c-maas-api-tls\") pod \"maas-api-8fd8f878b-dwvxp\" (UID: \"ff0b889f-0e6d-4954-9c3d-e534fe15075c\") " pod="opendatahub/maas-api-8fd8f878b-dwvxp"
May 28 17:28:18.445746 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.445752    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x8clx\" (UniqueName: \"kubernetes.io/projected/ff0b889f-0e6d-4954-9c3d-e534fe15075c-kube-api-access-x8clx\") pod \"maas-api-8fd8f878b-dwvxp\" (UID: \"ff0b889f-0e6d-4954-9c3d-e534fe15075c\") " pod="opendatahub/maas-api-8fd8f878b-dwvxp"
May 28 17:28:18.448265 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.448237    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff0b889f-0e6d-4954-9c3d-e534fe15075c-maas-api-tls\") pod \"maas-api-8fd8f878b-dwvxp\" (UID: \"ff0b889f-0e6d-4954-9c3d-e534fe15075c\") " pod="opendatahub/maas-api-8fd8f878b-dwvxp"
May 28 17:28:18.455406 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.455371    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8clx\" (UniqueName: \"kubernetes.io/projected/ff0b889f-0e6d-4954-9c3d-e534fe15075c-kube-api-access-x8clx\") pod \"maas-api-8fd8f878b-dwvxp\" (UID: \"ff0b889f-0e6d-4954-9c3d-e534fe15075c\") " pod="opendatahub/maas-api-8fd8f878b-dwvxp"
May 28 17:28:18.583014 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.582979    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-8fd8f878b-dwvxp"
May 28 17:28:18.629001 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.628966    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-5wb55"]
May 28 17:28:18.633195 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.633164    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-5wb55"
May 28 17:28:18.636268 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.636238    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
May 28 17:28:18.636553 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.636529    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-g26bt\""
May 28 17:28:18.643299 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.643023    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-5wb55"]
May 28 17:28:18.722536 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.722498    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-8fd8f878b-dwvxp"]
May 28 17:28:18.727864 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:28:18.727816    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff0b889f_0e6d_4954_9c3d_e534fe15075c.slice/crio-2274b98888ee4490a96c2ae430c41ca889f2ccf6a5e5653a612fa0c1c15b57f5 WatchSource:0}: Error finding container 2274b98888ee4490a96c2ae430c41ca889f2ccf6a5e5653a612fa0c1c15b57f5: Status 404 returned error can't find the container with id 2274b98888ee4490a96c2ae430c41ca889f2ccf6a5e5653a612fa0c1c15b57f5
May 28 17:28:18.749795 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.749763    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dzbq\" (UniqueName: \"kubernetes.io/projected/341d79c8-1cd0-4368-8b56-cb016d80f7e4-kube-api-access-9dzbq\") pod \"payload-processing-58b6c8fdc7-5wb55\" (UID: \"341d79c8-1cd0-4368-8b56-cb016d80f7e4\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-5wb55"
May 28 17:28:18.851007 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.850913    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9dzbq\" (UniqueName: \"kubernetes.io/projected/341d79c8-1cd0-4368-8b56-cb016d80f7e4-kube-api-access-9dzbq\") pod \"payload-processing-58b6c8fdc7-5wb55\" (UID: \"341d79c8-1cd0-4368-8b56-cb016d80f7e4\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-5wb55"
May 28 17:28:18.860315 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.860289    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dzbq\" (UniqueName: \"kubernetes.io/projected/341d79c8-1cd0-4368-8b56-cb016d80f7e4-kube-api-access-9dzbq\") pod \"payload-processing-58b6c8fdc7-5wb55\" (UID: \"341d79c8-1cd0-4368-8b56-cb016d80f7e4\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-5wb55"
May 28 17:28:18.949951 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:18.949911    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-5wb55"
May 28 17:28:19.086317 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:19.086291    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-5wb55"]
May 28 17:28:19.088695 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:28:19.088647    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod341d79c8_1cd0_4368_8b56_cb016d80f7e4.slice/crio-b10e796cc17b9dfe5dee1ea4339a893d774c86cd68ab37e99deb2882b5721b7a WatchSource:0}: Error finding container b10e796cc17b9dfe5dee1ea4339a893d774c86cd68ab37e99deb2882b5721b7a: Status 404 returned error can't find the container with id b10e796cc17b9dfe5dee1ea4339a893d774c86cd68ab37e99deb2882b5721b7a
May 28 17:28:19.276699 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:19.276582    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-8fd8f878b-dwvxp" event={"ID":"ff0b889f-0e6d-4954-9c3d-e534fe15075c","Type":"ContainerStarted","Data":"2274b98888ee4490a96c2ae430c41ca889f2ccf6a5e5653a612fa0c1c15b57f5"}
May 28 17:28:19.277670 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:19.277639    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-5wb55" event={"ID":"341d79c8-1cd0-4368-8b56-cb016d80f7e4","Type":"ContainerStarted","Data":"b10e796cc17b9dfe5dee1ea4339a893d774c86cd68ab37e99deb2882b5721b7a"}
May 28 17:28:21.285517 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:21.285420    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-5wb55" event={"ID":"341d79c8-1cd0-4368-8b56-cb016d80f7e4","Type":"ContainerStarted","Data":"53ea74ac9220ecc7ed9a741f768364b23c0f6b6ec6e592e975fa95eaa46acf56"}
May 28 17:28:21.285517 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:21.285487    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-5wb55"
May 28 17:28:21.286657 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:21.286635    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-8fd8f878b-dwvxp" event={"ID":"ff0b889f-0e6d-4954-9c3d-e534fe15075c","Type":"ContainerStarted","Data":"b305533df9146c884ae6204b76b30f3b0e79172e7476d8700a3d8ae249604874"}
May 28 17:28:21.286779 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:21.286771    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-8fd8f878b-dwvxp"
May 28 17:28:21.305908 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:21.305773    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58b6c8fdc7-5wb55" podStartSLOduration=1.564531658 podStartE2EDuration="3.305756965s" podCreationTimestamp="2026-05-28 17:28:18 +0000 UTC" firstStartedPulling="2026-05-28 17:28:19.090571192 +0000 UTC m=+663.330121562" lastFinishedPulling="2026-05-28 17:28:20.8317965 +0000 UTC m=+665.071346869" observedRunningTime="2026-05-28 17:28:21.304497059 +0000 UTC m=+665.544047451" watchObservedRunningTime="2026-05-28 17:28:21.305756965 +0000 UTC m=+665.545307353"
May 28 17:28:21.323632 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:21.323591    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-8fd8f878b-dwvxp" podStartSLOduration=1.223444649 podStartE2EDuration="3.323579073s" podCreationTimestamp="2026-05-28 17:28:18 +0000 UTC" firstStartedPulling="2026-05-28 17:28:18.729380356 +0000 UTC m=+662.968930723" lastFinishedPulling="2026-05-28 17:28:20.82951478 +0000 UTC m=+665.069065147" observedRunningTime="2026-05-28 17:28:21.322230284 +0000 UTC m=+665.561780684" watchObservedRunningTime="2026-05-28 17:28:21.323579073 +0000 UTC m=+665.563129463"
May 28 17:28:27.294802 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:27.294772    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-8fd8f878b-dwvxp"
May 28 17:28:32.292772 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:32.292742    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-5wb55"
May 28 17:28:47.540709 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.540655    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"]
May 28 17:28:47.544302 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.544281    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.548450 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.548424    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
May 28 17:28:47.548580 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.548451    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
May 28 17:28:47.548580 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.548425    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
May 28 17:28:47.548580 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.548425    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pmsz5\""
May 28 17:28:47.554565 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.554544    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"]
May 28 17:28:47.707411 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.707369    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4jgt\" (UniqueName: \"kubernetes.io/projected/83f6111a-26a2-49ab-a5dd-b4886780248e-kube-api-access-z4jgt\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.707610 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.707447    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/83f6111a-26a2-49ab-a5dd-b4886780248e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.707610 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.707513    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83f6111a-26a2-49ab-a5dd-b4886780248e-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.707610 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.707560    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83f6111a-26a2-49ab-a5dd-b4886780248e-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.707610 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.707592    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83f6111a-26a2-49ab-a5dd-b4886780248e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.707791 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.707660    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83f6111a-26a2-49ab-a5dd-b4886780248e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.808333 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.808304    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/83f6111a-26a2-49ab-a5dd-b4886780248e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.808505 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.808358    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83f6111a-26a2-49ab-a5dd-b4886780248e-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.808505 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.808394    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83f6111a-26a2-49ab-a5dd-b4886780248e-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.808505 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.808415    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83f6111a-26a2-49ab-a5dd-b4886780248e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.808505 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.808448    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83f6111a-26a2-49ab-a5dd-b4886780248e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.808505 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.808474    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z4jgt\" (UniqueName: \"kubernetes.io/projected/83f6111a-26a2-49ab-a5dd-b4886780248e-kube-api-access-z4jgt\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.808830 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.808802    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/83f6111a-26a2-49ab-a5dd-b4886780248e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.809066 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.808874    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83f6111a-26a2-49ab-a5dd-b4886780248e-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.809066 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.808951    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83f6111a-26a2-49ab-a5dd-b4886780248e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.811093 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.811065    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83f6111a-26a2-49ab-a5dd-b4886780248e-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.811287 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.811272    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83f6111a-26a2-49ab-a5dd-b4886780248e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.817217 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.817197    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4jgt\" (UniqueName: \"kubernetes.io/projected/83f6111a-26a2-49ab-a5dd-b4886780248e-kube-api-access-z4jgt\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-2q7nz\" (UID: \"83f6111a-26a2-49ab-a5dd-b4886780248e\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.855093 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.855067    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:28:47.994610 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:47.994577    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"]
May 28 17:28:47.998308 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:28:47.998276    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83f6111a_26a2_49ab_a5dd_b4886780248e.slice/crio-836303baf9330535d13fdd3b44ad8ca22c5fb252ca52204c88c5c074249ad5fb WatchSource:0}: Error finding container 836303baf9330535d13fdd3b44ad8ca22c5fb252ca52204c88c5c074249ad5fb: Status 404 returned error can't find the container with id 836303baf9330535d13fdd3b44ad8ca22c5fb252ca52204c88c5c074249ad5fb
May 28 17:28:48.379687 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:48.379630    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz" event={"ID":"83f6111a-26a2-49ab-a5dd-b4886780248e","Type":"ContainerStarted","Data":"836303baf9330535d13fdd3b44ad8ca22c5fb252ca52204c88c5c074249ad5fb"}
May 28 17:28:54.405771 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:54.405722    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz" event={"ID":"83f6111a-26a2-49ab-a5dd-b4886780248e","Type":"ContainerStarted","Data":"1df2e1b3129015b518346a2b1fa040e762ace78047dfc8be212d7b7783b457a6"}
May 28 17:28:59.427584 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:59.427493    2561 generic.go:358] "Generic (PLEG): container finished" podID="83f6111a-26a2-49ab-a5dd-b4886780248e" containerID="1df2e1b3129015b518346a2b1fa040e762ace78047dfc8be212d7b7783b457a6" exitCode=0
May 28 17:28:59.427584 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:28:59.427560    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz" event={"ID":"83f6111a-26a2-49ab-a5dd-b4886780248e","Type":"ContainerDied","Data":"1df2e1b3129015b518346a2b1fa040e762ace78047dfc8be212d7b7783b457a6"}
May 28 17:29:01.436291 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:29:01.436252    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz" event={"ID":"83f6111a-26a2-49ab-a5dd-b4886780248e","Type":"ContainerStarted","Data":"3ee385969d1c040e06bc1200820caf291dd1f54b2c9db96ee61f2bc774121921"}
May 28 17:29:01.436775 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:29:01.436477    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:29:01.458144 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:29:01.458097    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz" podStartSLOduration=1.8929140709999999 podStartE2EDuration="14.458078593s" podCreationTimestamp="2026-05-28 17:28:47 +0000 UTC" firstStartedPulling="2026-05-28 17:28:48.000030835 +0000 UTC m=+692.239581223" lastFinishedPulling="2026-05-28 17:29:00.565195376 +0000 UTC m=+704.804745745" observedRunningTime="2026-05-28 17:29:01.456370705 +0000 UTC m=+705.695921097" watchObservedRunningTime="2026-05-28 17:29:01.458078593 +0000 UTC m=+705.697628982"
May 28 17:29:12.453271 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:29:12.453229    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-2q7nz"
May 28 17:31:35.477995 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.477954    2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-865f658498-4g5z8"]
May 28 17:31:35.478483 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.478203    2561 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-865f658498-4g5z8" podUID="ed8ab7e4-40a1-430c-95d3-dd091dcae337" containerName="manager" containerID="cri-o://feca6fbc99ac95feaef28f2b87eb37b748360c2a4538abeec4773a22bc357809" gracePeriod=30
May 28 17:31:35.720822 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.720796    2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865f658498-4g5z8"
May 28 17:31:35.750368 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.750275    2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvfg8\" (UniqueName: \"kubernetes.io/projected/ed8ab7e4-40a1-430c-95d3-dd091dcae337-kube-api-access-rvfg8\") pod \"ed8ab7e4-40a1-430c-95d3-dd091dcae337\" (UID: \"ed8ab7e4-40a1-430c-95d3-dd091dcae337\") "
May 28 17:31:35.750512 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.750419    2561 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed8ab7e4-40a1-430c-95d3-dd091dcae337-webhook-cert\") pod \"ed8ab7e4-40a1-430c-95d3-dd091dcae337\" (UID: \"ed8ab7e4-40a1-430c-95d3-dd091dcae337\") "
May 28 17:31:35.752446 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.752414    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed8ab7e4-40a1-430c-95d3-dd091dcae337-kube-api-access-rvfg8" (OuterVolumeSpecName: "kube-api-access-rvfg8") pod "ed8ab7e4-40a1-430c-95d3-dd091dcae337" (UID: "ed8ab7e4-40a1-430c-95d3-dd091dcae337"). InnerVolumeSpecName "kube-api-access-rvfg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:31:35.752715 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.752659    2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed8ab7e4-40a1-430c-95d3-dd091dcae337-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ed8ab7e4-40a1-430c-95d3-dd091dcae337" (UID: "ed8ab7e4-40a1-430c-95d3-dd091dcae337"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 17:31:35.851153 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.851119    2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed8ab7e4-40a1-430c-95d3-dd091dcae337-webhook-cert\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\""
May 28 17:31:35.851153 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.851147    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvfg8\" (UniqueName: \"kubernetes.io/projected/ed8ab7e4-40a1-430c-95d3-dd091dcae337-kube-api-access-rvfg8\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\""
May 28 17:31:35.954775 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.954738    2561 generic.go:358] "Generic (PLEG): container finished" podID="ed8ab7e4-40a1-430c-95d3-dd091dcae337" containerID="feca6fbc99ac95feaef28f2b87eb37b748360c2a4538abeec4773a22bc357809" exitCode=0
May 28 17:31:35.954937 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.954811    2561 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865f658498-4g5z8"
May 28 17:31:35.954937 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.954827    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865f658498-4g5z8" event={"ID":"ed8ab7e4-40a1-430c-95d3-dd091dcae337","Type":"ContainerDied","Data":"feca6fbc99ac95feaef28f2b87eb37b748360c2a4538abeec4773a22bc357809"}
May 28 17:31:35.954937 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.954867    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865f658498-4g5z8" event={"ID":"ed8ab7e4-40a1-430c-95d3-dd091dcae337","Type":"ContainerDied","Data":"96c7e365ea7ebcd8800f70b649f63a9f1d52f222129c6c0074c5c2c448a358c1"}
May 28 17:31:35.954937 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.954882    2561 scope.go:117] "RemoveContainer" containerID="feca6fbc99ac95feaef28f2b87eb37b748360c2a4538abeec4773a22bc357809"
May 28 17:31:35.963172 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.963146    2561 scope.go:117] "RemoveContainer" containerID="feca6fbc99ac95feaef28f2b87eb37b748360c2a4538abeec4773a22bc357809"
May 28 17:31:35.963419 ip-10-0-139-178 kubenswrapper[2561]: E0528 17:31:35.963402    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feca6fbc99ac95feaef28f2b87eb37b748360c2a4538abeec4773a22bc357809\": container with ID starting with feca6fbc99ac95feaef28f2b87eb37b748360c2a4538abeec4773a22bc357809 not found: ID does not exist" containerID="feca6fbc99ac95feaef28f2b87eb37b748360c2a4538abeec4773a22bc357809"
May 28 17:31:35.963477 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.963427    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feca6fbc99ac95feaef28f2b87eb37b748360c2a4538abeec4773a22bc357809"} err="failed to get container status \"feca6fbc99ac95feaef28f2b87eb37b748360c2a4538abeec4773a22bc357809\": rpc error: code = NotFound desc = could not find container \"feca6fbc99ac95feaef28f2b87eb37b748360c2a4538abeec4773a22bc357809\": container with ID starting with feca6fbc99ac95feaef28f2b87eb37b748360c2a4538abeec4773a22bc357809 not found: ID does not exist"
May 28 17:31:35.978321 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.978294    2561 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-865f658498-4g5z8"]
May 28 17:31:35.982736 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:35.982713    2561 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-865f658498-4g5z8"]
May 28 17:31:36.321280 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:36.321247    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed8ab7e4-40a1-430c-95d3-dd091dcae337" path="/var/lib/kubelet/pods/ed8ab7e4-40a1-430c-95d3-dd091dcae337/volumes"
May 28 17:31:36.754759 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:36.754640    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-865f658498-zsggg"]
May 28 17:31:36.755119 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:36.755024    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed8ab7e4-40a1-430c-95d3-dd091dcae337" containerName="manager"
May 28 17:31:36.755119 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:36.755041    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed8ab7e4-40a1-430c-95d3-dd091dcae337" containerName="manager"
May 28 17:31:36.755119 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:36.755102    2561 memory_manager.go:356] "RemoveStaleState removing state" podUID="ed8ab7e4-40a1-430c-95d3-dd091dcae337" containerName="manager"
May 28 17:31:36.759298 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:36.759279    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865f658498-zsggg"
May 28 17:31:36.762489 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:36.762455    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bfnrd\""
May 28 17:31:36.762636 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:36.762524    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
May 28 17:31:36.763580 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:36.763561    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
May 28 17:31:36.768512 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:36.768483    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-865f658498-zsggg"]
May 28 17:31:36.860084 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:36.860044    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bf11062-cb0b-4864-956d-10aac6965a20-webhook-cert\") pod \"maas-controller-865f658498-zsggg\" (UID: \"8bf11062-cb0b-4864-956d-10aac6965a20\") " pod="opendatahub/maas-controller-865f658498-zsggg"
May 28 17:31:36.860253 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:36.860092    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrcf8\" (UniqueName: \"kubernetes.io/projected/8bf11062-cb0b-4864-956d-10aac6965a20-kube-api-access-rrcf8\") pod \"maas-controller-865f658498-zsggg\" (UID: \"8bf11062-cb0b-4864-956d-10aac6965a20\") " pod="opendatahub/maas-controller-865f658498-zsggg"
May 28 17:31:36.961165 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:36.961132    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bf11062-cb0b-4864-956d-10aac6965a20-webhook-cert\") pod \"maas-controller-865f658498-zsggg\" (UID: \"8bf11062-cb0b-4864-956d-10aac6965a20\") " pod="opendatahub/maas-controller-865f658498-zsggg"
May 28 17:31:36.961342 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:36.961170    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rrcf8\" (UniqueName: \"kubernetes.io/projected/8bf11062-cb0b-4864-956d-10aac6965a20-kube-api-access-rrcf8\") pod \"maas-controller-865f658498-zsggg\" (UID: \"8bf11062-cb0b-4864-956d-10aac6965a20\") " pod="opendatahub/maas-controller-865f658498-zsggg"
May 28 17:31:36.963760 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:36.963734    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bf11062-cb0b-4864-956d-10aac6965a20-webhook-cert\") pod \"maas-controller-865f658498-zsggg\" (UID: \"8bf11062-cb0b-4864-956d-10aac6965a20\") " pod="opendatahub/maas-controller-865f658498-zsggg"
May 28 17:31:36.971059 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:36.971032    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrcf8\" (UniqueName: \"kubernetes.io/projected/8bf11062-cb0b-4864-956d-10aac6965a20-kube-api-access-rrcf8\") pod \"maas-controller-865f658498-zsggg\" (UID: \"8bf11062-cb0b-4864-956d-10aac6965a20\") " pod="opendatahub/maas-controller-865f658498-zsggg"
May 28 17:31:37.070855 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:37.070815    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865f658498-zsggg"
May 28 17:31:37.207266 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:37.207238    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-865f658498-zsggg"]
May 28 17:31:37.209986 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:31:37.209956    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf11062_cb0b_4864_956d_10aac6965a20.slice/crio-aeb934066bc2782f3747dfeca03e41061f2443ae1898a247a352e9f0a8c4b4a2 WatchSource:0}: Error finding container aeb934066bc2782f3747dfeca03e41061f2443ae1898a247a352e9f0a8c4b4a2: Status 404 returned error can't find the container with id aeb934066bc2782f3747dfeca03e41061f2443ae1898a247a352e9f0a8c4b4a2
May 28 17:31:37.963424 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:37.963381    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865f658498-zsggg" event={"ID":"8bf11062-cb0b-4864-956d-10aac6965a20","Type":"ContainerStarted","Data":"07857df9a53abcb53c619aa4993e5ef34fc0cc39795f729fd8146822ddeed197"}
May 28 17:31:37.963424 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:37.963428    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865f658498-zsggg" event={"ID":"8bf11062-cb0b-4864-956d-10aac6965a20","Type":"ContainerStarted","Data":"aeb934066bc2782f3747dfeca03e41061f2443ae1898a247a352e9f0a8c4b4a2"}
May 28 17:31:37.963892 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:37.963530    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-865f658498-zsggg"
May 28 17:31:37.982393 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:37.982334    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-865f658498-zsggg" podStartSLOduration=1.403227328 podStartE2EDuration="1.982317965s" podCreationTimestamp="2026-05-28 17:31:36 +0000 UTC" firstStartedPulling="2026-05-28 17:31:37.211277794 +0000 UTC m=+861.450828162" lastFinishedPulling="2026-05-28 17:31:37.790368418 +0000 UTC m=+862.029918799" observedRunningTime="2026-05-28 17:31:37.981349483 +0000 UTC m=+862.220899910" watchObservedRunningTime="2026-05-28 17:31:37.982317965 +0000 UTC m=+862.221868398"
May 28 17:31:48.973665 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:31:48.973631    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-865f658498-zsggg"
May 28 17:53:02.784162 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:02.784087    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-jbd9m_c51e64e5-6eac-42a5-9533-cd8c059a542b/manager/0.log"
May 28 17:53:02.901551 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:02.901522    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-8fd8f878b-dwvxp_ff0b889f-0e6d-4954-9c3d-e534fe15075c/maas-api/0.log"
May 28 17:53:03.259293 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:03.259260    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-865f658498-zsggg_8bf11062-cb0b-4864-956d-10aac6965a20/manager/0.log"
May 28 17:53:03.382803 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:03.382770    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-ghngv_b596c9a0-b864-48a9-8e2d-50636502112d/manager/2.log"
May 28 17:53:03.744259 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:03.744227    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6795fd7c67-r6plx_9904e36d-4e36-4267-9781-a2d6f90c97e1/manager/0.log"
May 28 17:53:03.863158 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:03.863132    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-bwnfb_643025d4-e9cf-4dd2-a876-673ddf5fc2db/postgres/0.log"
May 28 17:53:05.244432 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:05.244402    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-wlmfh_d8d8d860-4b93-4af2-aec7-dfff155ae059/manager/0.log"
May 28 17:53:05.578017 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:05.577986    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-d52wt_f5ee3f0c-e6c4-4f1b-9876-d3dd8ab5c373/registry-server/0.log"
May 28 17:53:06.378872 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:06.378840    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-4c84k_f6c2939c-7bbc-407d-a977-afac48c78115/discovery/0.log"
May 28 17:53:06.488998 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:06.488966    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5454695dcd-kscnq_aba9bf3a-7748-495c-a1a2-f8884e127de7/kube-auth-proxy/0.log"
May 28 17:53:06.846120 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:06.846086    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-5wb55_341d79c8-1cd0-4368-8b56-cb016d80f7e4/payload-processing/0.log"
May 28 17:53:07.532142 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:07.532116    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-2q7nz_83f6111a-26a2-49ab-a5dd-b4886780248e/storage-initializer/0.log"
May 28 17:53:07.539967 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:07.539944    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-2q7nz_83f6111a-26a2-49ab-a5dd-b4886780248e/main/0.log"
May 28 17:53:15.080305 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:15.080269    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-trvkj_b878c489-e206-490c-954f-fc066b72599b/global-pull-secret-syncer/0.log"
May 28 17:53:15.201456 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:15.201428    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cvsnb_737fe4ef-9e14-4558-8696-2598182022ab/konnectivity-agent/0.log"
May 28 17:53:15.352012 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:15.351925    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-178.ec2.internal_8b6f5dedac2bb56dda3493221133ac3c/haproxy/0.log"
May 28 17:53:20.259375 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:20.259329    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-wlmfh_d8d8d860-4b93-4af2-aec7-dfff155ae059/manager/0.log"
May 28 17:53:20.451710 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:20.451659    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-d52wt_f5ee3f0c-e6c4-4f1b-9876-d3dd8ab5c373/registry-server/0.log"
May 28 17:53:22.935423 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:22.935395    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-57584b7c88-llqzq_31e62ced-916f-4f53-a789-64d173640f23/metrics-server/0.log"
May 28 17:53:23.166650 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.166624    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qgz26_308168d3-91ea-4330-9d54-e47e5d132813/node-exporter/0.log"
May 28 17:53:23.194854 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.194785    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qgz26_308168d3-91ea-4330-9d54-e47e5d132813/kube-rbac-proxy/0.log"
May 28 17:53:23.218877 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.218849    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qgz26_308168d3-91ea-4330-9d54-e47e5d132813/init-textfile/0.log"
May 28 17:53:23.765636 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.765599    2561 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"]
May 28 17:53:23.767967 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.767943    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:23.770829 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.770801    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7hsz9\"/\"openshift-service-ca.crt\""
May 28 17:53:23.772085 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.772059    2561 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7hsz9\"/\"kube-root-ca.crt\""
May 28 17:53:23.772085 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.772075    2561 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7hsz9\"/\"default-dockercfg-dlls8\""
May 28 17:53:23.779267 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.779241    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"]
May 28 17:53:23.888268 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.888228    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tctf\" (UniqueName: \"kubernetes.io/projected/ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811-kube-api-access-7tctf\") pod \"perf-node-gather-daemonset-2zjth\" (UID: \"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811\") " pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:23.888268 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.888272    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811-podres\") pod \"perf-node-gather-daemonset-2zjth\" (UID: \"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811\") " pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:23.888482 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.888341    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811-lib-modules\") pod \"perf-node-gather-daemonset-2zjth\" (UID: \"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811\") " pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:23.888482 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.888370    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811-sys\") pod \"perf-node-gather-daemonset-2zjth\" (UID: \"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811\") " pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:23.888482 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.888412    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811-proc\") pod \"perf-node-gather-daemonset-2zjth\" (UID: \"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811\") " pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:23.988933 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.988898    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7tctf\" (UniqueName: \"kubernetes.io/projected/ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811-kube-api-access-7tctf\") pod \"perf-node-gather-daemonset-2zjth\" (UID: \"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811\") " pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:23.988933 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.988937    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811-podres\") pod \"perf-node-gather-daemonset-2zjth\" (UID: \"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811\") " pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:23.989435 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.988971    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811-lib-modules\") pod \"perf-node-gather-daemonset-2zjth\" (UID: \"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811\") " pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:23.989435 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.989005    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811-sys\") pod \"perf-node-gather-daemonset-2zjth\" (UID: \"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811\") " pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:23.989435 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.989040    2561 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811-proc\") pod \"perf-node-gather-daemonset-2zjth\" (UID: \"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811\") " pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:23.989435 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.989084    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811-podres\") pod \"perf-node-gather-daemonset-2zjth\" (UID: \"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811\") " pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:23.989435 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.989134    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811-proc\") pod \"perf-node-gather-daemonset-2zjth\" (UID: \"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811\") " pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:23.989435 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.989135    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811-sys\") pod \"perf-node-gather-daemonset-2zjth\" (UID: \"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811\") " pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:23.989435 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:23.989168    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811-lib-modules\") pod \"perf-node-gather-daemonset-2zjth\" (UID: \"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811\") " pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:24.003607 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:24.003576    2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tctf\" (UniqueName: \"kubernetes.io/projected/ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811-kube-api-access-7tctf\") pod \"perf-node-gather-daemonset-2zjth\" (UID: \"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811\") " pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:24.078777 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:24.078733    2561 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:24.213140 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:24.213072    2561 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"]
May 28 17:53:24.215591 ip-10-0-139-178 kubenswrapper[2561]: W0528 17:53:24.215563    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podff7d88fd_d7c3_4c4a_8c3a_19926c9e1811.slice/crio-ba81829e8a05048fc4b2b6e0df800c9e84946a9cfa3f695fefa4c67c7f138685 WatchSource:0}: Error finding container ba81829e8a05048fc4b2b6e0df800c9e84946a9cfa3f695fefa4c67c7f138685: Status 404 returned error can't find the container with id ba81829e8a05048fc4b2b6e0df800c9e84946a9cfa3f695fefa4c67c7f138685
May 28 17:53:24.217209 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:24.217190    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 17:53:24.261117 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:24.261083    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth" event={"ID":"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811","Type":"ContainerStarted","Data":"ba81829e8a05048fc4b2b6e0df800c9e84946a9cfa3f695fefa4c67c7f138685"}
May 28 17:53:25.265502 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:25.265461    2561 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth" event={"ID":"ff7d88fd-d7c3-4c4a-8c3a-19926c9e1811","Type":"ContainerStarted","Data":"460622687f6ef81873715d17796832732cc2a993e404e92b31a9112550a3bdab"}
May 28 17:53:25.265912 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:25.265598    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:25.293065 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:25.293011    2561 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth" podStartSLOduration=2.292995397 podStartE2EDuration="2.292995397s" podCreationTimestamp="2026-05-28 17:53:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:53:25.292526082 +0000 UTC m=+2169.532076469" watchObservedRunningTime="2026-05-28 17:53:25.292995397 +0000 UTC m=+2169.532545786"
May 28 17:53:27.576163 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:27.576132    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jn46x_bac47327-8e2e-410d-9a4b-53d26f170d1b/dns/0.log"
May 28 17:53:27.600242 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:27.600202    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jn46x_bac47327-8e2e-410d-9a4b-53d26f170d1b/kube-rbac-proxy/0.log"
May 28 17:53:27.734427 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:27.734396    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vc2ht_4222bafb-9e5f-4884-832d-83364e16f3e9/dns-node-resolver/0.log"
May 28 17:53:28.250269 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:28.250237    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7467f99487-r96cs_440788b2-ab2b-413e-8150-482e0cb1b920/registry/0.log"
May 28 17:53:28.299149 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:28.299117    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kmg5m_cf00e197-64d4-4961-9730-248b6e1ec010/node-ca/0.log"
May 28 17:53:29.494144 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:29.494109    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-4c84k_f6c2939c-7bbc-407d-a977-afac48c78115/discovery/0.log"
May 28 17:53:29.525957 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:29.525923    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5454695dcd-kscnq_aba9bf3a-7748-495c-a1a2-f8884e127de7/kube-auth-proxy/0.log"
May 28 17:53:29.690947 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:29.690908    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-5wb55_341d79c8-1cd0-4368-8b56-cb016d80f7e4/payload-processing/0.log"
May 28 17:53:30.370472 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:30.370437    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9mdhg_7e64cb6c-a574-4272-acc4-5607085c3a2f/serve-healthcheck-canary/0.log"
May 28 17:53:31.256435 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:31.256404    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zbkcs_3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3/kube-rbac-proxy/0.log"
May 28 17:53:31.278721 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:31.278693    2561 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7hsz9/perf-node-gather-daemonset-2zjth"
May 28 17:53:31.286034 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:31.286005    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zbkcs_3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3/exporter/0.log"
May 28 17:53:31.320046 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:31.320021    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zbkcs_3ac46bd6-0a04-4f80-bc16-d38a2e8b76f3/extractor/0.log"
May 28 17:53:33.656534 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:33.656503    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-jbd9m_c51e64e5-6eac-42a5-9533-cd8c059a542b/manager/0.log"
May 28 17:53:33.707286 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:33.707257    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-8fd8f878b-dwvxp_ff0b889f-0e6d-4954-9c3d-e534fe15075c/maas-api/0.log"
May 28 17:53:33.873967 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:33.873925    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-865f658498-zsggg_8bf11062-cb0b-4864-956d-10aac6965a20/manager/0.log"
May 28 17:53:33.907012 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:33.906966    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-ghngv_b596c9a0-b864-48a9-8e2d-50636502112d/manager/1.log"
May 28 17:53:33.918187 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:33.918162    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-ghngv_b596c9a0-b864-48a9-8e2d-50636502112d/manager/2.log"
May 28 17:53:34.035847 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:34.035816    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6795fd7c67-r6plx_9904e36d-4e36-4267-9781-a2d6f90c97e1/manager/0.log"
May 28 17:53:34.063489 ip-10-0-139-178 kubenswrapper[2561]: I0528 17:53:34.063447    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-bwnfb_643025d4-e9cf-4dd2-a876-673ddf5fc2db/postgres/0.log"