Jun 03 16:33:50.345121 ip-10-0-140-123 systemd[1]: Starting Kubernetes Kubelet...
Jun 03 16:33:50.819035 ip-10-0-140-123 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 03 16:33:50.819035 ip-10-0-140-123 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jun 03 16:33:50.819035 ip-10-0-140-123 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 03 16:33:50.819035 ip-10-0-140-123 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jun 03 16:33:50.819035 ip-10-0-140-123 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 03 16:33:50.819926 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.819835    2569 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jun 03 16:33:50.822513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822498    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 03 16:33:50.822513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822513    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822517    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822520    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822523    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822526    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822529    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822532    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822536    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822540    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822543    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822545    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822548    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822551    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822554    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822556    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822559    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822562    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822564    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 03 16:33:50.822574 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822567    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822571    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822575    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822578    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822581    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822584    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822587    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822590    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822593    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822595    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822598    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822600    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822603    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822606    2569 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822608    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822611    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822613    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822616    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822618    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 03 16:33:50.823020 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822621    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822623    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822626    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822628    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822631    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822633    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822636    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822639    2569 feature_gate.go:328] unrecognized feature gate: Example
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822641    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822645    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822648    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822651    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822653    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822655    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822658    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822662    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822664    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822667    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822669    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822672    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 03 16:33:50.823499 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822675    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822677    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822680    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822682    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822684    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822687    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822690    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822692    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822695    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822698    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822700    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822703    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822705    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822708    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822710    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822713    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822715    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822718    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822721    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822724    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 03 16:33:50.824008 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822727    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822729    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822732    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822734    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822737    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822739    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822742    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.822745    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823141    2569 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823146    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823149    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823152    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823154    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823157    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823160    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823162    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823165    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823168    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823171    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823173    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 03 16:33:50.824496 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823176    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823179    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823181    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823184    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823187    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823189    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823192    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823194    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823197    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823206    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823209    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823212    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823215    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823218    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823221    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823223    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823226    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823229    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823232    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823236    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 03 16:33:50.825013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823239    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823241    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823244    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823247    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823249    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823252    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823254    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823258    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823260    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823262    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823265    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823267    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823270    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823273    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823275    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823278    2569 feature_gate.go:328] unrecognized feature gate: Example
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823280    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823282    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823285    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823287    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 03 16:33:50.825513 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823290    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823292    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823295    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823297    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823300    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823303    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823305    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823308    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823311    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823313    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823316    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823319    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823324    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823327    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823332    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823335    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823339    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823342    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823345    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 03 16:33:50.826018 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823347    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823350    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823353    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823355    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823358    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823361    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823363    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823366    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823369    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823372    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823374    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823377    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823379    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823382    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.823384    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824088    2569 flags.go:64] FLAG: --address="0.0.0.0"
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824100    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824107    2569 flags.go:64] FLAG: --anonymous-auth="true"
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824111    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824116    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824120    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jun 03 16:33:50.826505 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824124    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824129    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824133    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824136    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824140    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824143    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824146    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824149    2569 flags.go:64] FLAG: --cgroup-root=""
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824152    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824155    2569 flags.go:64] FLAG: --client-ca-file=""
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824158    2569 flags.go:64] FLAG: --cloud-config=""
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824161    2569 flags.go:64] FLAG: --cloud-provider="external"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824164    2569 flags.go:64] FLAG: --cluster-dns="[]"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824168    2569 flags.go:64] FLAG: --cluster-domain=""
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824178    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824181    2569 flags.go:64] FLAG: --config-dir=""
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824184    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824187    2569 flags.go:64] FLAG: --container-log-max-files="5"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824191    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824194    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824197    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824201    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824204    2569 flags.go:64] FLAG: --contention-profiling="false"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824207    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jun 03 16:33:50.827030 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824210    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824213    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824217    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824221    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824224    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824227    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824230    2569 flags.go:64] FLAG: --enable-load-reader="false"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824234    2569 flags.go:64] FLAG: --enable-server="true"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824237    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824242    2569 flags.go:64] FLAG: --event-burst="100"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824245    2569 flags.go:64] FLAG: --event-qps="50"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824248    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824251    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824254    2569 flags.go:64] FLAG: --eviction-hard=""
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824258    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824260    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824263    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824266    2569 flags.go:64] FLAG: --eviction-soft=""
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824269    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824272    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824275    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824278    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824282    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824285    2569 flags.go:64] FLAG: --fail-swap-on="true"
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824288    2569 flags.go:64] FLAG: --feature-gates=""
Jun 03 16:33:50.827625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824292    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824295    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824298    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824302    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824305    2569 flags.go:64] FLAG: --healthz-port="10248"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824308    2569 flags.go:64] FLAG: --help="false"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824311    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-140-123.ec2.internal"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824314    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824317    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824320    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824324    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824327    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824330    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824333    2569 flags.go:64] FLAG: --image-service-endpoint=""
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824336    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824340    2569 flags.go:64] FLAG: --kube-api-burst="100"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824343    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824347    2569 flags.go:64] FLAG: --kube-api-qps="50"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824349    2569 flags.go:64] FLAG: --kube-reserved=""
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824352    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824355    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824358    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824361    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824364    2569 flags.go:64] FLAG: --lock-file=""
Jun 03 16:33:50.828278 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824367    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824369    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824372    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824377    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824380    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824383    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824386    2569 flags.go:64] FLAG: --logging-format="text"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824393    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824397    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824400    2569 flags.go:64] FLAG: --manifest-url=""
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824410    2569 flags.go:64] FLAG: --manifest-url-header=""
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824416    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824419    2569 flags.go:64] FLAG: --max-open-files="1000000"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824423    2569 flags.go:64] FLAG: --max-pods="110"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824426    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824429    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824432    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824435    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824438    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824440    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824443    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824450    2569 flags.go:64] FLAG: --node-status-max-images="50"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824453    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824456    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Jun 03 16:33:50.828927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824460    2569 flags.go:64] FLAG: --pod-cidr=""
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824463    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824469    2569 flags.go:64] FLAG: --pod-manifest-path=""
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824472    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824475    2569 flags.go:64] FLAG: --pods-per-core="0"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824478    2569 flags.go:64] FLAG: --port="10250"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824482    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824484    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0044fda2b34104270"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824488    2569 flags.go:64] FLAG: --qos-reserved=""
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824491    2569 flags.go:64] FLAG: --read-only-port="10255"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824494    2569 flags.go:64] FLAG: --register-node="true"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824497    2569 flags.go:64] FLAG: --register-schedulable="true"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824500    2569 flags.go:64] FLAG: --register-with-taints=""
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824504    2569 flags.go:64] FLAG: --registry-burst="10"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824507    2569 flags.go:64] FLAG: --registry-qps="5"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824510    2569 flags.go:64] FLAG: --reserved-cpus=""
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824515    2569 flags.go:64] FLAG: --reserved-memory=""
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824519    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824523    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824525    2569 flags.go:64] FLAG: --rotate-certificates="false"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824528    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824532    2569 flags.go:64] FLAG: --runonce="false"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824535    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824538    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824541    2569 flags.go:64] FLAG: --seccomp-default="false"
Jun 03 16:33:50.829537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824543    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824546    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824549    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824553    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824556    2569 flags.go:64] FLAG: --storage-driver-password="root"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824558    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824561    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824565    2569 flags.go:64] FLAG: --storage-driver-user="root"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824568    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824571    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824574    2569 flags.go:64] FLAG: --system-cgroups=""
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824577    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824583    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824586    2569 flags.go:64] FLAG: --tls-cert-file=""
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824589    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824593    2569 flags.go:64] FLAG: --tls-min-version=""
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824596    2569 flags.go:64] FLAG: --tls-private-key-file=""
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824599    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824602    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824605    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824608    2569 flags.go:64] FLAG: --v="2"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824612    2569 flags.go:64] FLAG: --version="false"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824616    2569 flags.go:64] FLAG: --vmodule=""
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824622    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.824625    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jun 03 16:33:50.830237 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824717    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824721    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824725    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824728    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824731    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824733    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824736    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824739    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824742    2569 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824744    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824747    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824749    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824752    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824755    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824758    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824760    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824764    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824766    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824770    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 03 16:33:50.830893 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824774    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824777    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824779    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824782    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824785    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824787    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824790    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824804    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824807    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824809    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824812    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824817    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824819    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824822    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824825    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824827    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824830    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824832    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824835    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824838    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 03 16:33:50.831346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824841    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824845    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824848    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824852    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824855    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824857    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824860    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824865    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824868    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824870    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824874    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824876    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824879    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824882    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824884    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824886    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824889    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824892    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824894    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 03 16:33:50.831890 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824897    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824899    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824902    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824904    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824908    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824911    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824913    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824916    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824918    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824921    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824923    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824926    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824928    2569 feature_gate.go:328] unrecognized feature gate: Example
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824931    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824933    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824936    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824939    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824941    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824944    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824946    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 03 16:33:50.832582 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824951    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 03 16:33:50.833372 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824953    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 03 16:33:50.833372 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824956    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 03 16:33:50.833372 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824959    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 03 16:33:50.833372 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824962    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 03 16:33:50.833372 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824964    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 03 16:33:50.833372 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824967    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 03 16:33:50.833372 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.824969    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 03 16:33:50.833372 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.825608    2569 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Jun 03 16:33:50.833643 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.833626    2569 server.go:530] "Kubelet version" kubeletVersion="v1.33.12"
Jun 03 16:33:50.833677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.833644    2569 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jun 03 16:33:50.833708 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833692    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 03 16:33:50.833708 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833697    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 03 16:33:50.833708 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833701    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 03 16:33:50.833708 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833703    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 03 16:33:50.833708 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833707    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 03 16:33:50.833708 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833709    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833713    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833716    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833719    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833722    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833724    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833727    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833729    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833732    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833735    2569 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833737    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833740    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833743    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833746    2569 feature_gate.go:328] unrecognized feature gate: Example
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833749    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833752    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833754    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833757    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833760    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833763    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 03 16:33:50.833875 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833766    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833768    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833771    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833773    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833775    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833778    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833781    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833783    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833786    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833788    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833808    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833812    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833815    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833818    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833821    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833824    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833827    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833830    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833833    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833836    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 03 16:33:50.834380 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833839    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833842    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833844    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833847    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833850    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833852    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833855    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833857    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833860    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833862    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833865    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833868    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833872    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833876    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833879    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833881    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833884    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833887    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833890    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 03 16:33:50.834969 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833893    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833895    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833898    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833901    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833903    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833907    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833912    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833915    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833918    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833921    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833924    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833927    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833931    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833934    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833936    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833939    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833942    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833944    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833947    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833950    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 03 16:33:50.835454 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833952    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jun 03 16:33:50.835974 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.833955    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 03 16:33:50.835974 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.833960    2569 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Jun 03 16:33:50.835974 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834054    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 03 16:33:50.835974 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834059    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 03 16:33:50.835974 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834062    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 03 16:33:50.835974 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834065    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 03 16:33:50.835974 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834068    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 03 16:33:50.835974 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834071    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 03 16:33:50.835974 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834073    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 03 16:33:50.835974 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834076    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 03 16:33:50.835974 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834080    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 03 16:33:50.835974 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834082    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 03 16:33:50.835974 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834085    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 03 16:33:50.835974 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834088    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 03 16:33:50.835974 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834090    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834093    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834096    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834099    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834102    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834105    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834108    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834110    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834113    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834116    2569 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834119    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834122    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834126    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834129    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834132    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834135    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834138    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834142    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834145    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 03 16:33:50.836346 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834148    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834151    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834153    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834156    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834159    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834161    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834164    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834166    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834169    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834171    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834174    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834176    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834179    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834181    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834185    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834187    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834190    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834193    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834195    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834198    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834200    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 03 16:33:50.836905 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834203    2569 feature_gate.go:328] unrecognized feature gate: Example
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834205    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834208    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834211    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834214    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834216    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834219    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834221    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834224    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834226    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834228    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834231    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834234    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834236    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834239    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834241    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834244    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834246    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834249    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 03 16:33:50.837463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834251    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 03 16:33:50.837951 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834254    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 03 16:33:50.837951 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834257    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 03 16:33:50.837951 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834259    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 03 16:33:50.837951 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834263    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 03 16:33:50.837951 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834265    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 03 16:33:50.837951 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834267    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 03 16:33:50.837951 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834270    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 03 16:33:50.837951 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834272    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 03 16:33:50.837951 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834275    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 03 16:33:50.837951 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834278    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 03 16:33:50.837951 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834280    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 03 16:33:50.837951 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834283    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 03 16:33:50.837951 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834285    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 03 16:33:50.837951 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:50.834288    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 03 16:33:50.837951 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.834292    2569 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Jun 03 16:33:50.837951 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.834910    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Jun 03 16:33:50.838854 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.838841    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jun 03 16:33:50.839867 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.839855    2569 server.go:1019] "Starting client certificate rotation"
Jun 03 16:33:50.839969 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.839955    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jun 03 16:33:50.840006 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.839992    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jun 03 16:33:50.866783 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.866765    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jun 03 16:33:50.871505 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.871484    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jun 03 16:33:50.882194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.882174    2569 log.go:25] "Validated CRI v1 runtime API"
Jun 03 16:33:50.887810 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.887782    2569 log.go:25] "Validated CRI v1 image API"
Jun 03 16:33:50.889008 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.888992    2569 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jun 03 16:33:50.892805 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.892774    2569 fs.go:135] Filesystem UUIDs: map[01a480d2-392e-4f93-b427-8df52191286c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9152fe59-38ae-4db4-80ad-472019075a89:/dev/nvme0n1p4]
Jun 03 16:33:50.892859 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.892808    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jun 03 16:33:50.899049 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.898939    2569 manager.go:217] Machine: {Timestamp:2026-06-03 16:33:50.898419207 +0000 UTC m=+0.428938781 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100538 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21a1bacfee8d308f015a99f56b4471 SystemUUID:ec21a1ba-cfee-8d30-8f01-5a99f56b4471 BootID:de9b422c-6c2d-447b-8292-b06c7b7a2e7d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:22:fd:47:41:0d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:22:fd:47:41:0d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:1f:ed:f3:ab:9b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jun 03 16:33:50.899049 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.899038    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jun 03 16:33:50.899201 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.899114    2569 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jun 03 16:33:50.900268 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.900246    2569 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jun 03 16:33:50.900467 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.900270    2569 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-123.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jun 03 16:33:50.900519 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.900482    2569 topology_manager.go:138] "Creating topology manager with none policy"
Jun 03 16:33:50.900519 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.900495    2569 container_manager_linux.go:306] "Creating device plugin manager"
Jun 03 16:33:50.900571 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.900534    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jun 03 16:33:50.900571 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.900536    2569 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jun 03 16:33:50.900571 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.900554    2569 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jun 03 16:33:50.901317 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.901306    2569 state_mem.go:36] "Initialized new in-memory state store"
Jun 03 16:33:50.901582 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.901573    2569 server.go:1267] "Using root directory" path="/var/lib/kubelet"
Jun 03 16:33:50.904075 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.904064    2569 kubelet.go:491] "Attempting to sync node with API server"
Jun 03 16:33:50.904118 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.904078    2569 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
Jun 03 16:33:50.904118 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.904091    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jun 03 16:33:50.904118 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.904100    2569 kubelet.go:397] "Adding apiserver pod source"
Jun 03 16:33:50.904118 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.904108    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jun 03 16:33:50.905360 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.905348    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jun 03 16:33:50.905417 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.905369    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jun 03 16:33:50.908757 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.908742    2569 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1"
Jun 03 16:33:50.910891 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.910877    2569 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jun 03 16:33:50.912933 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.912916    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jun 03 16:33:50.912933 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.912940    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jun 03 16:33:50.912933 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.912947    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jun 03 16:33:50.913078 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.912952    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jun 03 16:33:50.913078 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.912961    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jun 03 16:33:50.913078 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.912967    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jun 03 16:33:50.913078 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.912974    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jun 03 16:33:50.913078 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.912979    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jun 03 16:33:50.913078 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.912985    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jun 03 16:33:50.913078 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.912991    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jun 03 16:33:50.913078 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.912999    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jun 03 16:33:50.913078 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.913004    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jun 03 16:33:50.914846 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.914834    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jun 03 16:33:50.914898 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.914849    2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jun 03 16:33:50.918048 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:50.918029    2569 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jun 03 16:33:50.918116 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:50.918031    2569 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jun 03 16:33:50.918266 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.918255    2569 watchdog_linux.go:99] "Systemd watchdog is not enabled"
Jun 03 16:33:50.918303 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.918291    2569 server.go:1295] "Started kubelet"
Jun 03 16:33:50.919047 ip-10-0-140-123 systemd[1]: Started Kubernetes Kubelet.
Jun 03 16:33:50.919265 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.919174    2569 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jun 03 16:33:50.919344 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.919332    2569 server_v1.go:47] "podresources" method="list" useActivePods=true
Jun 03 16:33:50.919951 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.919928    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jun 03 16:33:50.920305 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.920279    2569 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jun 03 16:33:50.922500 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.922481    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gvlc4"
Jun 03 16:33:50.922719 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.922706    2569 server.go:317] "Adding debug handlers to kubelet server"
Jun 03 16:33:50.924524 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.924508    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jun 03 16:33:50.924610 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.924518    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jun 03 16:33:50.925095 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.924687    2569 volume_manager.go:295] "The desired_state_of_world populator starts"
Jun 03 16:33:50.925095 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.924701    2569 volume_manager.go:297] "Starting Kubelet Volume Manager"
Jun 03 16:33:50.925095 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.924745    2569 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
Jun 03 16:33:50.925095 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.925029    2569 reconstruct.go:97] "Volume reconstruction finished"
Jun 03 16:33:50.925095 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.925037    2569 reconciler.go:26] "Reconciler: start to sync state"
Jun 03 16:33:50.925406 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.925388    2569 factory.go:55] Registering systemd factory
Jun 03 16:33:50.925467 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.925413    2569 factory.go:223] Registration of the systemd container factory successfully
Jun 03 16:33:50.925777 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:50.925680    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-123.ec2.internal\" not found"
Jun 03 16:33:50.926179 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.926161    2569 factory.go:153] Registering CRI-O factory
Jun 03 16:33:50.926179 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.926181    2569 factory.go:223] Registration of the crio container factory successfully
Jun 03 16:33:50.926328 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.926250    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jun 03 16:33:50.927539 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.927522    2569 factory.go:103] Registering Raw factory
Jun 03 16:33:50.927841 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.927830    2569 manager.go:1196] Started watching for new ooms in manager
Jun 03 16:33:50.928394 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.928377    2569 manager.go:319] Starting recovery of all containers
Jun 03 16:33:50.928786 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.928753    2569 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-123.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 03 16:33:50.928884 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:50.928846    2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jun 03 16:33:50.933929 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.933890    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gvlc4"
Jun 03 16:33:50.934249 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:50.929231    2569 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jun 03 16:33:50.934471 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:50.928969    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-123.ec2.internal.18b59f58ade46cc0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-123.ec2.internal,UID:ip-10-0-140-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-123.ec2.internal,},FirstTimestamp:2026-06-03 16:33:50.918266048 +0000 UTC m=+0.448785622,LastTimestamp:2026-06-03 16:33:50.918266048 +0000 UTC m=+0.448785622,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-123.ec2.internal,}"
Jun 03 16:33:50.935390 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:50.935350    2569 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jun 03 16:33:50.945486 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.945329    2569 manager.go:324] Recovery completed
Jun 03 16:33:50.949476 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.949464    2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 03 16:33:50.951379 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.951361    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasSufficientMemory"
Jun 03 16:33:50.951456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.951389    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasNoDiskPressure"
Jun 03 16:33:50.951456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.951399    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasSufficientPID"
Jun 03 16:33:50.951905 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.951890    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jun 03 16:33:50.951905 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.951905    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jun 03 16:33:50.952018 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.951923    2569 state_mem.go:36] "Initialized new in-memory state store"
Jun 03 16:33:50.954034 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.954020    2569 policy_none.go:49] "None policy: Start"
Jun 03 16:33:50.954076 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.954043    2569 memory_manager.go:186] "Starting memorymanager" policy="None"
Jun 03 16:33:50.954104 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.954087    2569 state_mem.go:35] "Initializing new in-memory state store"
Jun 03 16:33:50.987451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.987433    2569 manager.go:341] "Starting Device Plugin manager"
Jun 03 16:33:50.992449 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:50.987545    2569 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jun 03 16:33:50.992449 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.987557    2569 server.go:85] "Starting device plugin registration server"
Jun 03 16:33:50.992449 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.987852    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Jun 03 16:33:50.992449 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.987870    2569 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jun 03 16:33:50.992449 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.988030    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jun 03 16:33:50.992449 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.988129    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jun 03 16:33:50.992449 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:50.988138    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jun 03 16:33:50.992449 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:50.988560    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jun 03 16:33:50.992449 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:50.988592    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-123.ec2.internal\" not found"
Jun 03 16:33:51.066388 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.066357    2569 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
Jun 03 16:33:51.067599 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.067582    2569 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
Jun 03 16:33:51.067652 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.067608    2569 status_manager.go:230] "Starting to sync pod status with apiserver"
Jun 03 16:33:51.067652 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.067630    2569 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jun 03 16:33:51.067652 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.067640    2569 kubelet.go:2451] "Starting kubelet main sync loop"
Jun 03 16:33:51.067772 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:51.067694    2569 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jun 03 16:33:51.070629 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.070582    2569 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 03 16:33:51.089055 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.089028    2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 03 16:33:51.090122 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.090107    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasSufficientMemory"
Jun 03 16:33:51.090194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.090134    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasNoDiskPressure"
Jun 03 16:33:51.090194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.090144    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasSufficientPID"
Jun 03 16:33:51.090194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.090165    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.098498 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.098484    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.098537 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:51.098506    2569 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-123.ec2.internal\": node \"ip-10-0-140-123.ec2.internal\" not found"
Jun 03 16:33:51.116472 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:51.116441    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-123.ec2.internal\" not found"
Jun 03 16:33:51.167821 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.167780    2569 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-123.ec2.internal"]
Jun 03 16:33:51.167946 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.167867    2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 03 16:33:51.169709 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.169695    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasSufficientMemory"
Jun 03 16:33:51.169768 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.169721    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasNoDiskPressure"
Jun 03 16:33:51.169768 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.169735    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasSufficientPID"
Jun 03 16:33:51.170822 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.170809    2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 03 16:33:51.170945 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.170931    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.170990 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.170961    2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 03 16:33:51.171653 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.171635    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasSufficientMemory"
Jun 03 16:33:51.171744 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.171660    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasNoDiskPressure"
Jun 03 16:33:51.171744 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.171639    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasSufficientMemory"
Jun 03 16:33:51.171744 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.171668    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasSufficientPID"
Jun 03 16:33:51.171744 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.171687    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasNoDiskPressure"
Jun 03 16:33:51.171744 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.171699    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasSufficientPID"
Jun 03 16:33:51.172704 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.172692    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.172758 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.172716    2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 03 16:33:51.173670 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.173655    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasSufficientMemory"
Jun 03 16:33:51.173755 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.173683    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasNoDiskPressure"
Jun 03 16:33:51.173755 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.173696    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeHasSufficientPID"
Jun 03 16:33:51.202411 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:51.202393    2569 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-123.ec2.internal\" not found" node="ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.206890 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:51.206876    2569 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-123.ec2.internal\" not found" node="ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.216604 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:51.216586    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-123.ec2.internal\" not found"
Jun 03 16:33:51.226057 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.226035    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9cddf9626608f1bc5c1a3ac95a08d15-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal\" (UID: \"a9cddf9626608f1bc5c1a3ac95a08d15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.226137 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.226063    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9ef217c3fb12636ad0c2fc98457b1b27-config\") pod \"kube-apiserver-proxy-ip-10-0-140-123.ec2.internal\" (UID: \"9ef217c3fb12636ad0c2fc98457b1b27\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.226137 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.226079    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a9cddf9626608f1bc5c1a3ac95a08d15-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal\" (UID: \"a9cddf9626608f1bc5c1a3ac95a08d15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.317053 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:51.317009    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-123.ec2.internal\" not found"
Jun 03 16:33:51.326331 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.326271    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a9cddf9626608f1bc5c1a3ac95a08d15-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal\" (UID: \"a9cddf9626608f1bc5c1a3ac95a08d15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.326331 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.326302    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9cddf9626608f1bc5c1a3ac95a08d15-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal\" (UID: \"a9cddf9626608f1bc5c1a3ac95a08d15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.326331 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.326325    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9ef217c3fb12636ad0c2fc98457b1b27-config\") pod \"kube-apiserver-proxy-ip-10-0-140-123.ec2.internal\" (UID: \"9ef217c3fb12636ad0c2fc98457b1b27\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.326484 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.326361    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9ef217c3fb12636ad0c2fc98457b1b27-config\") pod \"kube-apiserver-proxy-ip-10-0-140-123.ec2.internal\" (UID: \"9ef217c3fb12636ad0c2fc98457b1b27\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.326484 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.326380    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a9cddf9626608f1bc5c1a3ac95a08d15-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal\" (UID: \"a9cddf9626608f1bc5c1a3ac95a08d15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.326484 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.326387    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9cddf9626608f1bc5c1a3ac95a08d15-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal\" (UID: \"a9cddf9626608f1bc5c1a3ac95a08d15\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.417687 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:51.417648    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-123.ec2.internal\" not found"
Jun 03 16:33:51.505167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.505132    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.509788 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.509770    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-123.ec2.internal"
Jun 03 16:33:51.518377 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:51.518360    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-123.ec2.internal\" not found"
Jun 03 16:33:51.618952 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:51.618867    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-123.ec2.internal\" not found"
Jun 03 16:33:51.719336 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:51.719298    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-123.ec2.internal\" not found"
Jun 03 16:33:51.768382 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.768359    2569 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 03 16:33:51.819949 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:51.819910    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-123.ec2.internal\" not found"
Jun 03 16:33:51.839346 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.839328    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jun 03 16:33:51.839499 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.839482    2569 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jun 03 16:33:51.839543 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.839522    2569 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jun 03 16:33:51.920966 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:51.920907    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-123.ec2.internal\" not found"
Jun 03 16:33:51.925090 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.925073    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jun 03 16:33:51.936869 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.936836    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-02 16:28:50 +0000 UTC" deadline="2027-11-07 13:22:02.395205339 +0000 UTC"
Jun 03 16:33:51.936869 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.936861    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12524h48m10.45834797s"
Jun 03 16:33:51.947109 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.947089    2569 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jun 03 16:33:51.956189 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:51.956167    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9cddf9626608f1bc5c1a3ac95a08d15.slice/crio-ad0077f06f3ff78a0163854a028552cf1f98134bb08382a3cb382847fd6efa57 WatchSource:0}: Error finding container ad0077f06f3ff78a0163854a028552cf1f98134bb08382a3cb382847fd6efa57: Status 404 returned error can't find the container with id ad0077f06f3ff78a0163854a028552cf1f98134bb08382a3cb382847fd6efa57
Jun 03 16:33:51.956374 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:51.956359    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ef217c3fb12636ad0c2fc98457b1b27.slice/crio-19514e74dc19577084d0340ba3719f502ec913cb3677622bf3a53d3d76af7936 WatchSource:0}: Error finding container 19514e74dc19577084d0340ba3719f502ec913cb3677622bf3a53d3d76af7936: Status 404 returned error can't find the container with id 19514e74dc19577084d0340ba3719f502ec913cb3677622bf3a53d3d76af7936
Jun 03 16:33:51.960358 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.960340    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 03 16:33:51.969225 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.969206    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7mv4j"
Jun 03 16:33:51.974125 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:51.974110    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7mv4j"
Jun 03 16:33:52.021995 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:52.021965    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-123.ec2.internal\" not found"
Jun 03 16:33:52.071253 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.071206    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-123.ec2.internal" event={"ID":"9ef217c3fb12636ad0c2fc98457b1b27","Type":"ContainerStarted","Data":"19514e74dc19577084d0340ba3719f502ec913cb3677622bf3a53d3d76af7936"}
Jun 03 16:33:52.072181 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.072141    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal" event={"ID":"a9cddf9626608f1bc5c1a3ac95a08d15","Type":"ContainerStarted","Data":"ad0077f06f3ff78a0163854a028552cf1f98134bb08382a3cb382847fd6efa57"}
Jun 03 16:33:52.122325 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:52.122302    2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-123.ec2.internal\" not found"
Jun 03 16:33:52.135532 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.135512    2569 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 03 16:33:52.167757 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.167735    2569 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 03 16:33:52.225274 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.225213    2569 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-123.ec2.internal"
Jun 03 16:33:52.236374 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.236352    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jun 03 16:33:52.237660 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.237646    2569 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal"
Jun 03 16:33:52.247230 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.247215    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jun 03 16:33:52.882669 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.882639    2569 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 03 16:33:52.904994 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.904967    2569 apiserver.go:52] "Watching apiserver"
Jun 03 16:33:52.913365 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.913347    2569 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jun 03 16:33:52.915353 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.915326    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-vzp5g","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal","openshift-network-diagnostics/network-check-target-bs44f","kube-system/konnectivity-agent-j5pvb","kube-system/kube-apiserver-proxy-ip-10-0-140-123.ec2.internal","openshift-image-registry/node-ca-ccbzr","openshift-multus/multus-additional-cni-plugins-qmmz5","openshift-multus/multus-s2z68","openshift-multus/network-metrics-daemon-q8x4r","openshift-network-operator/iptables-alerter-bmrbz","openshift-ovn-kubernetes/ovnkube-node-xh2c4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj","openshift-cluster-node-tuning-operator/tuned-xz9hl"]
Jun 03 16:33:52.917233 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.917216    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:52.918173 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.918152    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:33:52.918268 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:52.918217    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:33:52.919184 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.919122    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j5pvb"
Jun 03 16:33:52.919720 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.919700    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jun 03 16:33:52.920052 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.920030    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jun 03 16:33:52.920052 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.920045    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jun 03 16:33:52.920206 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.920095    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-s9mwr\""
Jun 03 16:33:52.920206 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.920146    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jun 03 16:33:52.920414 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.920387    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jun 03 16:33:52.921588 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.921190    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ccbzr"
Jun 03 16:33:52.921670 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.921587    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jun 03 16:33:52.922300 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.922277    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jun 03 16:33:52.922382 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.922298    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6jscc\""
Jun 03 16:33:52.923313 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.922658    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vzp5g"
Jun 03 16:33:52.924845 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.923613    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jun 03 16:33:52.924845 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.923864    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jun 03 16:33:52.924845 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.923980    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lqx45\""
Jun 03 16:33:52.924845 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.924099    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jun 03 16:33:52.926284 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.925859    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:33:52.926284 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.925900    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jun 03 16:33:52.926284 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:52.925930    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:33:52.926284 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.926166    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f9m8t\""
Jun 03 16:33:52.926284 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.926170    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jun 03 16:33:52.927387 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.927360    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.929312 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.929286    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bmrbz"
Jun 03 16:33:52.929463 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.929444    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:52.929856 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.929835    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jun 03 16:33:52.929958 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.929909    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bc6wv\""
Jun 03 16:33:52.930654 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.930638    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:52.932304 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.931968    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:52.932304 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.932216    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jun 03 16:33:52.932304 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.932237    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jun 03 16:33:52.932500 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.932323    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fjlgl\""
Jun 03 16:33:52.933069 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.932932    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jun 03 16:33:52.933069 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.932969    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4kzz9\""
Jun 03 16:33:52.933782 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.933761    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jun 03 16:33:52.933911 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.933841    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jun 03 16:33:52.934223 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.934200    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jun 03 16:33:52.934301 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.934260    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jun 03 16:33:52.934301 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.934199    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jun 03 16:33:52.934408 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.934377    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jun 03 16:33:52.934475 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.934455    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jun 03 16:33:52.934475 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.934467    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dgqsw\""
Jun 03 16:33:52.934605 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.934587    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jun 03 16:33:52.934779 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.934758    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8b6b06a-4893-46c5-b030-88a2c01abe88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:52.934875 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.934758    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jun 03 16:33:52.934875 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.934846    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c0fd9ccf-e728-4aba-9249-daed366df1e5-host\") pod \"node-ca-ccbzr\" (UID: \"c0fd9ccf-e728-4aba-9249-daed366df1e5\") " pod="openshift-image-registry/node-ca-ccbzr"
Jun 03 16:33:52.934970 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.934880    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fefcdfd2-e498-45e1-ae6d-fa8431ba97fa-hosts-file\") pod \"node-resolver-vzp5g\" (UID: \"fefcdfd2-e498-45e1-ae6d-fa8431ba97fa\") " pod="openshift-dns/node-resolver-vzp5g"
Jun 03 16:33:52.934970 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.934915    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fefcdfd2-e498-45e1-ae6d-fa8431ba97fa-tmp-dir\") pod \"node-resolver-vzp5g\" (UID: \"fefcdfd2-e498-45e1-ae6d-fa8431ba97fa\") " pod="openshift-dns/node-resolver-vzp5g"
Jun 03 16:33:52.934970 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.934962    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a5d651cc-b55b-4806-96ef-1e93043eb8e0-cni-binary-copy\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.935116 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.934980    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-var-lib-cni-bin\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.935116 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935009    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-etc-kubernetes\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.935116 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935033    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8b6b06a-4893-46c5-b030-88a2c01abe88-cnibin\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:52.935116 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935052    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8b6b06a-4893-46c5-b030-88a2c01abe88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:52.935116 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935072    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrp7z\" (UniqueName: \"kubernetes.io/projected/d1292e61-8cac-4672-af45-d434487b3426-kube-api-access-zrp7z\") pod \"network-metrics-daemon-q8x4r\" (UID: \"d1292e61-8cac-4672-af45-d434487b3426\") " pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:33:52.935116 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935108    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-run-netns\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.935456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935126    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-hostroot\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.935456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935143    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a5d651cc-b55b-4806-96ef-1e93043eb8e0-multus-daemon-config\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.935456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935166    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cklgv\" (UniqueName: \"kubernetes.io/projected/c0fd9ccf-e728-4aba-9249-daed366df1e5-kube-api-access-cklgv\") pod \"node-ca-ccbzr\" (UID: \"c0fd9ccf-e728-4aba-9249-daed366df1e5\") " pod="openshift-image-registry/node-ca-ccbzr"
Jun 03 16:33:52.935456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935192    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64rjq\" (UniqueName: \"kubernetes.io/projected/fefcdfd2-e498-45e1-ae6d-fa8431ba97fa-kube-api-access-64rjq\") pod \"node-resolver-vzp5g\" (UID: \"fefcdfd2-e498-45e1-ae6d-fa8431ba97fa\") " pod="openshift-dns/node-resolver-vzp5g"
Jun 03 16:33:52.935456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935208    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-system-cni-dir\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.935456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935224    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-multus-socket-dir-parent\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.935456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935246    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkzhq\" (UniqueName: \"kubernetes.io/projected/a5d651cc-b55b-4806-96ef-1e93043eb8e0-kube-api-access-jkzhq\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.935456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935267    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-run-k8s-cni-cncf-io\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.935456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935309    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-run-multus-certs\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.935456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935350    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8b6b06a-4893-46c5-b030-88a2c01abe88-cni-binary-copy\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:52.935456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935359    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jun 03 16:33:52.935456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935387    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c9mf\" (UniqueName: \"kubernetes.io/projected/f8b6b06a-4893-46c5-b030-88a2c01abe88-kube-api-access-4c9mf\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:52.935456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935423    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fpcsj\""
Jun 03 16:33:52.935456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935430    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-multus-conf-dir\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.935456 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935461    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8b6b06a-4893-46c5-b030-88a2c01abe88-system-cni-dir\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:52.936194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935484    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jun 03 16:33:52.936194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935489    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs\") pod \"network-metrics-daemon-q8x4r\" (UID: \"d1292e61-8cac-4672-af45-d434487b3426\") " pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:33:52.936194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935511    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5023b960-cc21-4146-bab5-a63abb187a37-konnectivity-ca\") pod \"konnectivity-agent-j5pvb\" (UID: \"5023b960-cc21-4146-bab5-a63abb187a37\") " pod="kube-system/konnectivity-agent-j5pvb"
Jun 03 16:33:52.936194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935545    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-multus-cni-dir\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.936194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935569    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-var-lib-cni-multus\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.936194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935593    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-var-lib-kubelet\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.936194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935626    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8b6b06a-4893-46c5-b030-88a2c01abe88-os-release\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:52.936194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935652    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8b6b06a-4893-46c5-b030-88a2c01abe88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:52.936194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935676    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c0fd9ccf-e728-4aba-9249-daed366df1e5-serviceca\") pod \"node-ca-ccbzr\" (UID: \"c0fd9ccf-e728-4aba-9249-daed366df1e5\") " pod="openshift-image-registry/node-ca-ccbzr"
Jun 03 16:33:52.936194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935718    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5023b960-cc21-4146-bab5-a63abb187a37-agent-certs\") pod \"konnectivity-agent-j5pvb\" (UID: \"5023b960-cc21-4146-bab5-a63abb187a37\") " pod="kube-system/konnectivity-agent-j5pvb"
Jun 03 16:33:52.936194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935747    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-cnibin\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.936194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935787    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-os-release\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:52.936194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.935839    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfz5b\" (UniqueName: \"kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b\") pod \"network-check-target-bs44f\" (UID: \"cad8c088-209d-4676-b925-be649bcf3f51\") " pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:33:52.974617 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.974587    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 16:28:51 +0000 UTC" deadline="2028-01-14 05:16:40.74021928 +0000 UTC"
Jun 03 16:33:52.974617 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:52.974618    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14148h42m47.76560441s"
Jun 03 16:33:53.025885 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.025854    2569 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
Jun 03 16:33:53.036034 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036002    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-var-lib-cni-multus\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.036182 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036045    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8b6b06a-4893-46c5-b030-88a2c01abe88-os-release\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.036182 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036079    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-etc-selinux\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.036182 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036107    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-sys-fs\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.036182 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036129    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-tuned\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.036182 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036131    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-var-lib-cni-multus\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.036182 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036152    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-run-ovn\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.036182 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036168    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8b6b06a-4893-46c5-b030-88a2c01abe88-os-release\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.036182 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036184    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5023b960-cc21-4146-bab5-a63abb187a37-agent-certs\") pod \"konnectivity-agent-j5pvb\" (UID: \"5023b960-cc21-4146-bab5-a63abb187a37\") " pod="kube-system/konnectivity-agent-j5pvb"
Jun 03 16:33:53.036649 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036230    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qfz5b\" (UniqueName: \"kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b\") pod \"network-check-target-bs44f\" (UID: \"cad8c088-209d-4676-b925-be649bcf3f51\") " pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:33:53.036649 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036390    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.036649 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036426    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08a04603-3ecf-4148-96c1-21ca6a2a75bf-tmp\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.036649 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036459    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8b6b06a-4893-46c5-b030-88a2c01abe88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.036649 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036485    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c0fd9ccf-e728-4aba-9249-daed366df1e5-host\") pod \"node-ca-ccbzr\" (UID: \"c0fd9ccf-e728-4aba-9249-daed366df1e5\") " pod="openshift-image-registry/node-ca-ccbzr"
Jun 03 16:33:53.036649 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036547    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c0fd9ccf-e728-4aba-9249-daed366df1e5-host\") pod \"node-ca-ccbzr\" (UID: \"c0fd9ccf-e728-4aba-9249-daed366df1e5\") " pod="openshift-image-registry/node-ca-ccbzr"
Jun 03 16:33:53.036649 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036572    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fefcdfd2-e498-45e1-ae6d-fa8431ba97fa-tmp-dir\") pod \"node-resolver-vzp5g\" (UID: \"fefcdfd2-e498-45e1-ae6d-fa8431ba97fa\") " pod="openshift-dns/node-resolver-vzp5g"
Jun 03 16:33:53.036649 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036592    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jun 03 16:33:53.036649 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036629    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-sys\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.037086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036675    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-var-lib-kubelet\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.037086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036699    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzmjf\" (UniqueName: \"kubernetes.io/projected/aab5d542-5157-42c9-9cca-2190dbc377bb-kube-api-access-pzmjf\") pod \"iptables-alerter-bmrbz\" (UID: \"aab5d542-5157-42c9-9cca-2190dbc377bb\") " pod="openshift-network-operator/iptables-alerter-bmrbz"
Jun 03 16:33:53.037086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036717    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-run-netns\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.037086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036735    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-var-lib-cni-bin\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.037086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036761    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8b6b06a-4893-46c5-b030-88a2c01abe88-cnibin\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.037086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036781    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zrp7z\" (UniqueName: \"kubernetes.io/projected/d1292e61-8cac-4672-af45-d434487b3426-kube-api-access-zrp7z\") pod \"network-metrics-daemon-q8x4r\" (UID: \"d1292e61-8cac-4672-af45-d434487b3426\") " pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:33:53.037086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036818    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-run-systemd\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.037086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036867    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-var-lib-cni-bin\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.037086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036898    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bbeda011-ea40-4374-b7bf-cb93cae95d56-ovn-node-metrics-cert\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.037086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036897    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fefcdfd2-e498-45e1-ae6d-fa8431ba97fa-tmp-dir\") pod \"node-resolver-vzp5g\" (UID: \"fefcdfd2-e498-45e1-ae6d-fa8431ba97fa\") " pod="openshift-dns/node-resolver-vzp5g"
Jun 03 16:33:53.037086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036914    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bbeda011-ea40-4374-b7bf-cb93cae95d56-ovnkube-script-lib\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.037086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036954    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a5d651cc-b55b-4806-96ef-1e93043eb8e0-multus-daemon-config\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.037086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.036981    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8b6b06a-4893-46c5-b030-88a2c01abe88-cnibin\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.037086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037027    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cklgv\" (UniqueName: \"kubernetes.io/projected/c0fd9ccf-e728-4aba-9249-daed366df1e5-kube-api-access-cklgv\") pod \"node-ca-ccbzr\" (UID: \"c0fd9ccf-e728-4aba-9249-daed366df1e5\") " pod="openshift-image-registry/node-ca-ccbzr"
Jun 03 16:33:53.037086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037039    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8b6b06a-4893-46c5-b030-88a2c01abe88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.037086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037051    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-64rjq\" (UniqueName: \"kubernetes.io/projected/fefcdfd2-e498-45e1-ae6d-fa8431ba97fa-kube-api-access-64rjq\") pod \"node-resolver-vzp5g\" (UID: \"fefcdfd2-e498-45e1-ae6d-fa8431ba97fa\") " pod="openshift-dns/node-resolver-vzp5g"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037118    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhtpp\" (UniqueName: \"kubernetes.io/projected/08a04603-3ecf-4148-96c1-21ca6a2a75bf-kube-api-access-nhtpp\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037282    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-slash\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037311    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-cni-bin\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037340    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-multus-socket-dir-parent\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037367    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-device-dir\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037391    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-systemd-units\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037417    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-run-k8s-cni-cncf-io\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037452    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-run-multus-certs\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037453    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-multus-socket-dir-parent\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037482    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-run-k8s-cni-cncf-io\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037487    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8b6b06a-4893-46c5-b030-88a2c01abe88-cni-binary-copy\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037514    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-run-multus-certs\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037533    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4c9mf\" (UniqueName: \"kubernetes.io/projected/f8b6b06a-4893-46c5-b030-88a2c01abe88-kube-api-access-4c9mf\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037562    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8b6b06a-4893-46c5-b030-88a2c01abe88-system-cni-dir\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037578    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a5d651cc-b55b-4806-96ef-1e93043eb8e0-multus-daemon-config\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037591    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-lib-modules\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.037849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037626    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n4dj\" (UniqueName: \"kubernetes.io/projected/bbeda011-ea40-4374-b7bf-cb93cae95d56-kube-api-access-7n4dj\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037666    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-systemd\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037690    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5023b960-cc21-4146-bab5-a63abb187a37-konnectivity-ca\") pod \"konnectivity-agent-j5pvb\" (UID: \"5023b960-cc21-4146-bab5-a63abb187a37\") " pod="kube-system/konnectivity-agent-j5pvb"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037712    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-multus-cni-dir\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037759    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8b6b06a-4893-46c5-b030-88a2c01abe88-system-cni-dir\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037768    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-multus-cni-dir\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037784    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-var-lib-kubelet\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037839    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8b6b06a-4893-46c5-b030-88a2c01abe88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037850    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-var-lib-kubelet\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037907    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8b6b06a-4893-46c5-b030-88a2c01abe88-cni-binary-copy\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037922    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c0fd9ccf-e728-4aba-9249-daed366df1e5-serviceca\") pod \"node-ca-ccbzr\" (UID: \"c0fd9ccf-e728-4aba-9249-daed366df1e5\") " pod="openshift-image-registry/node-ca-ccbzr"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037963    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-826zk\" (UniqueName: \"kubernetes.io/projected/0655ed2c-0ff0-498e-af5a-eca49b7663e4-kube-api-access-826zk\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.037991    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-modprobe-d\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038019    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-cnibin\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038045    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-os-release\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038074    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aab5d542-5157-42c9-9cca-2190dbc377bb-iptables-alerter-script\") pod \"iptables-alerter-bmrbz\" (UID: \"aab5d542-5157-42c9-9cca-2190dbc377bb\") " pod="openshift-network-operator/iptables-alerter-bmrbz"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038100    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aab5d542-5157-42c9-9cca-2190dbc377bb-host-slash\") pod \"iptables-alerter-bmrbz\" (UID: \"aab5d542-5157-42c9-9cca-2190dbc377bb\") " pod="openshift-network-operator/iptables-alerter-bmrbz"
Jun 03 16:33:53.038677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038127    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-run-openvswitch\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038154    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-cni-netd\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038185    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fefcdfd2-e498-45e1-ae6d-fa8431ba97fa-hosts-file\") pod \"node-resolver-vzp5g\" (UID: \"fefcdfd2-e498-45e1-ae6d-fa8431ba97fa\") " pod="openshift-dns/node-resolver-vzp5g"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038213    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-sysctl-d\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038238    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-run\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038263    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-run-ovn-kubernetes\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038291    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a5d651cc-b55b-4806-96ef-1e93043eb8e0-cni-binary-copy\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038313    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-etc-kubernetes\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038340    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8b6b06a-4893-46c5-b030-88a2c01abe88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038361    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8b6b06a-4893-46c5-b030-88a2c01abe88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038373    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5023b960-cc21-4146-bab5-a63abb187a37-konnectivity-ca\") pod \"konnectivity-agent-j5pvb\" (UID: \"5023b960-cc21-4146-bab5-a63abb187a37\") " pod="kube-system/konnectivity-agent-j5pvb"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038365    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-socket-dir\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038418    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-registration-dir\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038422    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c0fd9ccf-e728-4aba-9249-daed366df1e5-serviceca\") pod \"node-ca-ccbzr\" (UID: \"c0fd9ccf-e728-4aba-9249-daed366df1e5\") " pod="openshift-image-registry/node-ca-ccbzr"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038452    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-cnibin\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038502    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fefcdfd2-e498-45e1-ae6d-fa8431ba97fa-hosts-file\") pod \"node-resolver-vzp5g\" (UID: \"fefcdfd2-e498-45e1-ae6d-fa8431ba97fa\") " pod="openshift-dns/node-resolver-vzp5g"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038508    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-etc-kubernetes\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.039479 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038519    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-var-lib-openvswitch\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038553    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-os-release\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038569    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038596    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bbeda011-ea40-4374-b7bf-cb93cae95d56-ovnkube-config\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038636    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8b6b06a-4893-46c5-b030-88a2c01abe88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038642    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-run-netns\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038684    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-hostroot\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038687    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-host-run-netns\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038733    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-hostroot\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038762    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-system-cni-dir\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038790    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jkzhq\" (UniqueName: \"kubernetes.io/projected/a5d651cc-b55b-4806-96ef-1e93043eb8e0-kube-api-access-jkzhq\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038848    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-host\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038854    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-system-cni-dir\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038875    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-etc-openvswitch\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038901    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-log-socket\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038924    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-sysconfig\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038947    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-sysctl-conf\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.040215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.038974    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-multus-conf-dir\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.040983 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.039000    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs\") pod \"network-metrics-daemon-q8x4r\" (UID: \"d1292e61-8cac-4672-af45-d434487b3426\") " pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:33:53.040983 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.039026    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-kubernetes\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.040983 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.039050    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-kubelet\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.040983 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.039071    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-node-log\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.040983 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.039097    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bbeda011-ea40-4374-b7bf-cb93cae95d56-env-overrides\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.040983 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:53.039114    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 16:33:53.040983 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.039147    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a5d651cc-b55b-4806-96ef-1e93043eb8e0-cni-binary-copy\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.040983 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:53.039192    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs podName:d1292e61-8cac-4672-af45-d434487b3426 nodeName:}" failed. No retries permitted until 2026-06-03 16:33:53.539160716 +0000 UTC m=+3.069680300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs") pod "network-metrics-daemon-q8x4r" (UID: "d1292e61-8cac-4672-af45-d434487b3426") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 16:33:53.040983 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.039209    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a5d651cc-b55b-4806-96ef-1e93043eb8e0-multus-conf-dir\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.040983 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.040524    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5023b960-cc21-4146-bab5-a63abb187a37-agent-certs\") pod \"konnectivity-agent-j5pvb\" (UID: \"5023b960-cc21-4146-bab5-a63abb187a37\") " pod="kube-system/konnectivity-agent-j5pvb"
Jun 03 16:33:53.047014 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:53.046993    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 03 16:33:53.047127 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:53.047022    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 03 16:33:53.047127 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:53.047036    2569 projected.go:194] Error preparing data for projected volume kube-api-access-qfz5b for pod openshift-network-diagnostics/network-check-target-bs44f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 16:33:53.047127 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:53.047112    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b podName:cad8c088-209d-4676-b925-be649bcf3f51 nodeName:}" failed. No retries permitted until 2026-06-03 16:33:53.54707627 +0000 UTC m=+3.077595834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qfz5b" (UniqueName: "kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b") pod "network-check-target-bs44f" (UID: "cad8c088-209d-4676-b925-be649bcf3f51") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 16:33:53.049474 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.049455    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cklgv\" (UniqueName: \"kubernetes.io/projected/c0fd9ccf-e728-4aba-9249-daed366df1e5-kube-api-access-cklgv\") pod \"node-ca-ccbzr\" (UID: \"c0fd9ccf-e728-4aba-9249-daed366df1e5\") " pod="openshift-image-registry/node-ca-ccbzr"
Jun 03 16:33:53.049578 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.049517    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkzhq\" (UniqueName: \"kubernetes.io/projected/a5d651cc-b55b-4806-96ef-1e93043eb8e0-kube-api-access-jkzhq\") pod \"multus-s2z68\" (UID: \"a5d651cc-b55b-4806-96ef-1e93043eb8e0\") " pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.049656 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.049635    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64rjq\" (UniqueName: \"kubernetes.io/projected/fefcdfd2-e498-45e1-ae6d-fa8431ba97fa-kube-api-access-64rjq\") pod \"node-resolver-vzp5g\" (UID: \"fefcdfd2-e498-45e1-ae6d-fa8431ba97fa\") " pod="openshift-dns/node-resolver-vzp5g"
Jun 03 16:33:53.050077 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.050032    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c9mf\" (UniqueName: \"kubernetes.io/projected/f8b6b06a-4893-46c5-b030-88a2c01abe88-kube-api-access-4c9mf\") pod \"multus-additional-cni-plugins-qmmz5\" (UID: \"f8b6b06a-4893-46c5-b030-88a2c01abe88\") " pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.050544 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.050519    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrp7z\" (UniqueName: \"kubernetes.io/projected/d1292e61-8cac-4672-af45-d434487b3426-kube-api-access-zrp7z\") pod \"network-metrics-daemon-q8x4r\" (UID: \"d1292e61-8cac-4672-af45-d434487b3426\") " pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:33:53.140192 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140122    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-sys-fs\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.140192 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140160    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-tuned\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.140192 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140177    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-run-ovn\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.140451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140211    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.140451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140233    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08a04603-3ecf-4148-96c1-21ca6a2a75bf-tmp\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.140451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140248    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-sys-fs\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.140451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140259    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-sys\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.140451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140304    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-var-lib-kubelet\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.140451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140312    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-sys\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.140451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140346    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pzmjf\" (UniqueName: \"kubernetes.io/projected/aab5d542-5157-42c9-9cca-2190dbc377bb-kube-api-access-pzmjf\") pod \"iptables-alerter-bmrbz\" (UID: \"aab5d542-5157-42c9-9cca-2190dbc377bb\") " pod="openshift-network-operator/iptables-alerter-bmrbz"
Jun 03 16:33:53.140451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140357    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-var-lib-kubelet\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.140451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140346    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-run-ovn\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.140451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140346    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.140451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140386    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-run-netns\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.140451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140364    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-run-netns\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.140451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140419    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-run-systemd\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.140451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140435    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bbeda011-ea40-4374-b7bf-cb93cae95d56-ovn-node-metrics-cert\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.140451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140458    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bbeda011-ea40-4374-b7bf-cb93cae95d56-ovnkube-script-lib\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140484    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nhtpp\" (UniqueName: \"kubernetes.io/projected/08a04603-3ecf-4148-96c1-21ca6a2a75bf-kube-api-access-nhtpp\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140504    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-run-systemd\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140546    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-slash\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140509    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-slash\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140584    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-cni-bin\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140611    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-device-dir\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140632    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-systemd-units\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140662    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-lib-modules\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140680    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-cni-bin\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140686    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7n4dj\" (UniqueName: \"kubernetes.io/projected/bbeda011-ea40-4374-b7bf-cb93cae95d56-kube-api-access-7n4dj\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140714    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-device-dir\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140734    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-systemd\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140770    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-826zk\" (UniqueName: \"kubernetes.io/projected/0655ed2c-0ff0-498e-af5a-eca49b7663e4-kube-api-access-826zk\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140775    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-systemd-units\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140814    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-modprobe-d\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140844    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aab5d542-5157-42c9-9cca-2190dbc377bb-iptables-alerter-script\") pod \"iptables-alerter-bmrbz\" (UID: \"aab5d542-5157-42c9-9cca-2190dbc377bb\") " pod="openshift-network-operator/iptables-alerter-bmrbz"
Jun 03 16:33:53.141167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140848    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-systemd\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140869    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aab5d542-5157-42c9-9cca-2190dbc377bb-host-slash\") pod \"iptables-alerter-bmrbz\" (UID: \"aab5d542-5157-42c9-9cca-2190dbc377bb\") " pod="openshift-network-operator/iptables-alerter-bmrbz"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140898    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-run-openvswitch\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140922    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-cni-netd\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140949    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-sysctl-d\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140972    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-run\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.140995    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-run-ovn-kubernetes\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141026    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-socket-dir\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141052    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-registration-dir\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141065    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-run-openvswitch\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141077    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-var-lib-openvswitch\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141114    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141150    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bbeda011-ea40-4374-b7bf-cb93cae95d56-ovnkube-config\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141183    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-host\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141208    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-etc-openvswitch\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141223    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bbeda011-ea40-4374-b7bf-cb93cae95d56-ovnkube-script-lib\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141233    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-log-socket\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.141955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141257    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-sysconfig\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141271    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aab5d542-5157-42c9-9cca-2190dbc377bb-host-slash\") pod \"iptables-alerter-bmrbz\" (UID: \"aab5d542-5157-42c9-9cca-2190dbc377bb\") " pod="openshift-network-operator/iptables-alerter-bmrbz"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141281    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-sysctl-conf\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141295    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-registration-dir\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141303    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-modprobe-d\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141335    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-kubernetes\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141357    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-cni-netd\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141362    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-kubelet\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141390    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-node-log\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141403    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-run-ovn-kubernetes\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141401    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-sysctl-d\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141401    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-sysctl-conf\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141439    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bbeda011-ea40-4374-b7bf-cb93cae95d56-env-overrides\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141458    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-var-lib-openvswitch\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141470    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-etc-selinux\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141491    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-socket-dir\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141025    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-lib-modules\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.142749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141513    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-log-socket\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.143619 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141512    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-host\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.143619 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141473    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-run\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.143619 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141552    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aab5d542-5157-42c9-9cca-2190dbc377bb-iptables-alerter-script\") pod \"iptables-alerter-bmrbz\" (UID: \"aab5d542-5157-42c9-9cca-2190dbc377bb\") " pod="openshift-network-operator/iptables-alerter-bmrbz"
Jun 03 16:33:53.143619 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141572    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-etc-openvswitch\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.143619 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141595    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-sysconfig\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.143619 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141605    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-kubelet\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.143619 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141616    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-kubernetes\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.143619 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141629    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-node-log\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.143619 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141638    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbeda011-ea40-4374-b7bf-cb93cae95d56-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.143619 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141692    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0655ed2c-0ff0-498e-af5a-eca49b7663e4-etc-selinux\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.143619 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.141998    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bbeda011-ea40-4374-b7bf-cb93cae95d56-ovnkube-config\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.143619 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.142004    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bbeda011-ea40-4374-b7bf-cb93cae95d56-env-overrides\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.143619 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.143454    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bbeda011-ea40-4374-b7bf-cb93cae95d56-ovn-node-metrics-cert\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.144107 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.143625    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08a04603-3ecf-4148-96c1-21ca6a2a75bf-etc-tuned\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.144107 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.143694    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08a04603-3ecf-4148-96c1-21ca6a2a75bf-tmp\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.150966 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.150930    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n4dj\" (UniqueName: \"kubernetes.io/projected/bbeda011-ea40-4374-b7bf-cb93cae95d56-kube-api-access-7n4dj\") pod \"ovnkube-node-xh2c4\" (UID: \"bbeda011-ea40-4374-b7bf-cb93cae95d56\") " pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.151150 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.151129    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhtpp\" (UniqueName: \"kubernetes.io/projected/08a04603-3ecf-4148-96c1-21ca6a2a75bf-kube-api-access-nhtpp\") pod \"tuned-xz9hl\" (UID: \"08a04603-3ecf-4148-96c1-21ca6a2a75bf\") " pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.151526 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.151501    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-826zk\" (UniqueName: \"kubernetes.io/projected/0655ed2c-0ff0-498e-af5a-eca49b7663e4-kube-api-access-826zk\") pod \"aws-ebs-csi-driver-node-9pgpj\" (UID: \"0655ed2c-0ff0-498e-af5a-eca49b7663e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.151676 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.151533    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzmjf\" (UniqueName: \"kubernetes.io/projected/aab5d542-5157-42c9-9cca-2190dbc377bb-kube-api-access-pzmjf\") pod \"iptables-alerter-bmrbz\" (UID: \"aab5d542-5157-42c9-9cca-2190dbc377bb\") " pod="openshift-network-operator/iptables-alerter-bmrbz"
Jun 03 16:33:53.230593 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.230560    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qmmz5"
Jun 03 16:33:53.237378 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.237355    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j5pvb"
Jun 03 16:33:53.247411 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.247395    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ccbzr"
Jun 03 16:33:53.252933 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.252915    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vzp5g"
Jun 03 16:33:53.260505 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.260489    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s2z68"
Jun 03 16:33:53.267136 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.267108    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bmrbz"
Jun 03 16:33:53.273738 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.273721    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:33:53.281353 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.281334    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj"
Jun 03 16:33:53.284929 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.284910    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xz9hl"
Jun 03 16:33:53.543494 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.543468    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs\") pod \"network-metrics-daemon-q8x4r\" (UID: \"d1292e61-8cac-4672-af45-d434487b3426\") " pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:33:53.543588 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:53.543579    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 16:33:53.543636 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:53.543627    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs podName:d1292e61-8cac-4672-af45-d434487b3426 nodeName:}" failed. No retries permitted until 2026-06-03 16:33:54.543613948 +0000 UTC m=+4.074133521 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs") pod "network-metrics-daemon-q8x4r" (UID: "d1292e61-8cac-4672-af45-d434487b3426") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 16:33:53.561366 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:53.561339    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0fd9ccf_e728_4aba_9249_daed366df1e5.slice/crio-3b151b0ace726e5ac48252383a177fbf3ceea7388f1628d637f8d84c01a63386 WatchSource:0}: Error finding container 3b151b0ace726e5ac48252383a177fbf3ceea7388f1628d637f8d84c01a63386: Status 404 returned error can't find the container with id 3b151b0ace726e5ac48252383a177fbf3ceea7388f1628d637f8d84c01a63386
Jun 03 16:33:53.562467 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:53.562435    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8b6b06a_4893_46c5_b030_88a2c01abe88.slice/crio-71df2b0d99815b0187126571eac6c329f1c9b41772f663405991ef88e043ed38 WatchSource:0}: Error finding container 71df2b0d99815b0187126571eac6c329f1c9b41772f663405991ef88e043ed38: Status 404 returned error can't find the container with id 71df2b0d99815b0187126571eac6c329f1c9b41772f663405991ef88e043ed38
Jun 03 16:33:53.563243 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:53.563201    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5023b960_cc21_4146_bab5_a63abb187a37.slice/crio-cc1b4c0ac4373e7b07ebabf1b9b34bf586f29428b3428ae3ffa2a8227d66cc34 WatchSource:0}: Error finding container cc1b4c0ac4373e7b07ebabf1b9b34bf586f29428b3428ae3ffa2a8227d66cc34: Status 404 returned error can't find the container with id cc1b4c0ac4373e7b07ebabf1b9b34bf586f29428b3428ae3ffa2a8227d66cc34
Jun 03 16:33:53.564066 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:53.563822    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a04603_3ecf_4148_96c1_21ca6a2a75bf.slice/crio-d5fefc582d6532ae8ae8c3c1b7a0facb82e7a8570cbbb01d604030f0a26e8dbc WatchSource:0}: Error finding container d5fefc582d6532ae8ae8c3c1b7a0facb82e7a8570cbbb01d604030f0a26e8dbc: Status 404 returned error can't find the container with id d5fefc582d6532ae8ae8c3c1b7a0facb82e7a8570cbbb01d604030f0a26e8dbc
Jun 03 16:33:53.566985 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:53.566967    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbeda011_ea40_4374_b7bf_cb93cae95d56.slice/crio-a932f0c960096970d9544b26a1006c387a0c913a82b6e6ea3289c3c335e7c596 WatchSource:0}: Error finding container a932f0c960096970d9544b26a1006c387a0c913a82b6e6ea3289c3c335e7c596: Status 404 returned error can't find the container with id a932f0c960096970d9544b26a1006c387a0c913a82b6e6ea3289c3c335e7c596
Jun 03 16:33:53.568045 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:53.568025    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5d651cc_b55b_4806_96ef_1e93043eb8e0.slice/crio-7745f3f78bc905d01ef6b5789ea66a14af28c066c185f25fed11625ecaf82ef1 WatchSource:0}: Error finding container 7745f3f78bc905d01ef6b5789ea66a14af28c066c185f25fed11625ecaf82ef1: Status 404 returned error can't find the container with id 7745f3f78bc905d01ef6b5789ea66a14af28c066c185f25fed11625ecaf82ef1
Jun 03 16:33:53.568688 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:33:53.568607    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaab5d542_5157_42c9_9cca_2190dbc377bb.slice/crio-0b1be40b5fdeed76554b0c168af0a1274c730283eb9074266d726e2e503680c6 WatchSource:0}: Error finding container 0b1be40b5fdeed76554b0c168af0a1274c730283eb9074266d726e2e503680c6: Status 404 returned error can't find the container with id 0b1be40b5fdeed76554b0c168af0a1274c730283eb9074266d726e2e503680c6
Jun 03 16:33:53.643951 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.643772    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qfz5b\" (UniqueName: \"kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b\") pod \"network-check-target-bs44f\" (UID: \"cad8c088-209d-4676-b925-be649bcf3f51\") " pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:33:53.644053 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:53.643898    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 03 16:33:53.644053 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:53.644008    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 03 16:33:53.644053 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:53.644023    2569 projected.go:194] Error preparing data for projected volume kube-api-access-qfz5b for pod openshift-network-diagnostics/network-check-target-bs44f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 16:33:53.644203 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:53.644074    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b podName:cad8c088-209d-4676-b925-be649bcf3f51 nodeName:}" failed. No retries permitted until 2026-06-03 16:33:54.644055584 +0000 UTC m=+4.174575146 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfz5b" (UniqueName: "kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b") pod "network-check-target-bs44f" (UID: "cad8c088-209d-4676-b925-be649bcf3f51") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 16:33:53.652575 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.652556    2569 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 03 16:33:53.975229 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.975160    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 16:28:51 +0000 UTC" deadline="2027-11-01 10:09:20.060040449 +0000 UTC"
Jun 03 16:33:53.975229 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:53.975198    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12377h35m26.084846608s"
Jun 03 16:33:54.069312 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:54.068718    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:33:54.069312 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:54.068857    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:33:54.080502 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:54.080423    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bmrbz" event={"ID":"aab5d542-5157-42c9-9cca-2190dbc377bb","Type":"ContainerStarted","Data":"0b1be40b5fdeed76554b0c168af0a1274c730283eb9074266d726e2e503680c6"}
Jun 03 16:33:54.088732 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:54.088634    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s2z68" event={"ID":"a5d651cc-b55b-4806-96ef-1e93043eb8e0","Type":"ContainerStarted","Data":"7745f3f78bc905d01ef6b5789ea66a14af28c066c185f25fed11625ecaf82ef1"}
Jun 03 16:33:54.108468 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:54.108432    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xz9hl" event={"ID":"08a04603-3ecf-4148-96c1-21ca6a2a75bf","Type":"ContainerStarted","Data":"d5fefc582d6532ae8ae8c3c1b7a0facb82e7a8570cbbb01d604030f0a26e8dbc"}
Jun 03 16:33:54.110991 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:54.110928    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j5pvb" event={"ID":"5023b960-cc21-4146-bab5-a63abb187a37","Type":"ContainerStarted","Data":"cc1b4c0ac4373e7b07ebabf1b9b34bf586f29428b3428ae3ffa2a8227d66cc34"}
Jun 03 16:33:54.116701 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:54.116674    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj" event={"ID":"0655ed2c-0ff0-498e-af5a-eca49b7663e4","Type":"ContainerStarted","Data":"248da4945c634994d11c6cd302e0beacf06cbd758e5cbcfcd6534da2716b1a43"}
Jun 03 16:33:54.120955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:54.120928    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vzp5g" event={"ID":"fefcdfd2-e498-45e1-ae6d-fa8431ba97fa","Type":"ContainerStarted","Data":"be1438f15630ad84b112efc6e488b92f062ba4d39ad91d94fce020ea8154e8f9"}
Jun 03 16:33:54.134789 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:54.134697    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4" event={"ID":"bbeda011-ea40-4374-b7bf-cb93cae95d56","Type":"ContainerStarted","Data":"a932f0c960096970d9544b26a1006c387a0c913a82b6e6ea3289c3c335e7c596"}
Jun 03 16:33:54.163844 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:54.163810    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmmz5" event={"ID":"f8b6b06a-4893-46c5-b030-88a2c01abe88","Type":"ContainerStarted","Data":"71df2b0d99815b0187126571eac6c329f1c9b41772f663405991ef88e043ed38"}
Jun 03 16:33:54.173545 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:54.173503    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ccbzr" event={"ID":"c0fd9ccf-e728-4aba-9249-daed366df1e5","Type":"ContainerStarted","Data":"3b151b0ace726e5ac48252383a177fbf3ceea7388f1628d637f8d84c01a63386"}
Jun 03 16:33:54.189079 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:54.189047    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-123.ec2.internal" event={"ID":"9ef217c3fb12636ad0c2fc98457b1b27","Type":"ContainerStarted","Data":"85aab4fdb69f8e8bc3e1b852571b8df51e1b4c8c0d38ebcd87f12a720a506e2f"}
Jun 03 16:33:54.207136 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:54.206643    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-123.ec2.internal" podStartSLOduration=2.206621346 podStartE2EDuration="2.206621346s" podCreationTimestamp="2026-06-03 16:33:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:33:54.205982955 +0000 UTC m=+3.736502540" watchObservedRunningTime="2026-06-03 16:33:54.206621346 +0000 UTC m=+3.737140930"
Jun 03 16:33:54.211309 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:54.211278    2569 generic.go:358] "Generic (PLEG): container finished" podID="a9cddf9626608f1bc5c1a3ac95a08d15" containerID="48ab846a2a08449310f96cba589449a0242524163ec44f492195124ed221bb2a" exitCode=0
Jun 03 16:33:54.211538 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:54.211322    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal" event={"ID":"a9cddf9626608f1bc5c1a3ac95a08d15","Type":"ContainerDied","Data":"48ab846a2a08449310f96cba589449a0242524163ec44f492195124ed221bb2a"}
Jun 03 16:33:54.551247 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:54.551200    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs\") pod \"network-metrics-daemon-q8x4r\" (UID: \"d1292e61-8cac-4672-af45-d434487b3426\") " pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:33:54.551430 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:54.551379    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 16:33:54.551494 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:54.551446    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs podName:d1292e61-8cac-4672-af45-d434487b3426 nodeName:}" failed. No retries permitted until 2026-06-03 16:33:56.551425879 +0000 UTC m=+6.081945454 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs") pod "network-metrics-daemon-q8x4r" (UID: "d1292e61-8cac-4672-af45-d434487b3426") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 16:33:54.652034 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:54.651946    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qfz5b\" (UniqueName: \"kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b\") pod \"network-check-target-bs44f\" (UID: \"cad8c088-209d-4676-b925-be649bcf3f51\") " pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:33:54.652197 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:54.652121    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 03 16:33:54.652197 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:54.652142    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 03 16:33:54.652197 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:54.652156    2569 projected.go:194] Error preparing data for projected volume kube-api-access-qfz5b for pod openshift-network-diagnostics/network-check-target-bs44f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 16:33:54.652362 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:54.652231    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b podName:cad8c088-209d-4676-b925-be649bcf3f51 nodeName:}" failed. No retries permitted until 2026-06-03 16:33:56.652212571 +0000 UTC m=+6.182732138 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfz5b" (UniqueName: "kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b") pod "network-check-target-bs44f" (UID: "cad8c088-209d-4676-b925-be649bcf3f51") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 16:33:55.070279 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:55.070251    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:33:55.070727 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:55.070394    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:33:55.229364 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:55.229329    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal" event={"ID":"a9cddf9626608f1bc5c1a3ac95a08d15","Type":"ContainerStarted","Data":"024c3e0bff3b3ce3cdec612d85af132758c293aff1355cf851100a9089dcc6f3"}
Jun 03 16:33:56.068490 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:56.068039    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:33:56.068490 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:56.068154    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:33:56.569343 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:56.568569    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs\") pod \"network-metrics-daemon-q8x4r\" (UID: \"d1292e61-8cac-4672-af45-d434487b3426\") " pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:33:56.569343 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:56.568811    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 16:33:56.569343 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:56.568875    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs podName:d1292e61-8cac-4672-af45-d434487b3426 nodeName:}" failed. No retries permitted until 2026-06-03 16:34:00.568857583 +0000 UTC m=+10.099377150 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs") pod "network-metrics-daemon-q8x4r" (UID: "d1292e61-8cac-4672-af45-d434487b3426") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 16:33:56.669669 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:56.669620    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qfz5b\" (UniqueName: \"kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b\") pod \"network-check-target-bs44f\" (UID: \"cad8c088-209d-4676-b925-be649bcf3f51\") " pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:33:56.669883 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:56.669862    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 03 16:33:56.669972 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:56.669893    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 03 16:33:56.669972 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:56.669906    2569 projected.go:194] Error preparing data for projected volume kube-api-access-qfz5b for pod openshift-network-diagnostics/network-check-target-bs44f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 16:33:56.670078 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:56.669986    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b podName:cad8c088-209d-4676-b925-be649bcf3f51 nodeName:}" failed. No retries permitted until 2026-06-03 16:34:00.66995692 +0000 UTC m=+10.200476482 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfz5b" (UniqueName: "kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b") pod "network-check-target-bs44f" (UID: "cad8c088-209d-4676-b925-be649bcf3f51") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 16:33:57.069453 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:57.069421    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:33:57.069643 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:57.069563    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:33:58.068124 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:58.067842    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:33:58.068124 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:58.067985    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:33:59.068586 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:59.068548    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:33:59.069081 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:59.068710    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:33:59.281423 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:59.281359    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-123.ec2.internal" podStartSLOduration=7.281341447 podStartE2EDuration="7.281341447s" podCreationTimestamp="2026-06-03 16:33:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:33:55.243987367 +0000 UTC m=+4.774506951" watchObservedRunningTime="2026-06-03 16:33:59.281341447 +0000 UTC m=+8.811861030"
Jun 03 16:33:59.281613 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:59.281550    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-cw9c7"]
Jun 03 16:33:59.283326 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:59.283305    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:33:59.283452 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:59.283385    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cw9c7" podUID="700242dd-718f-4d0f-9a09-410717e2a0fa"
Jun 03 16:33:59.390219 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:59.390130    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret\") pod \"global-pull-secret-syncer-cw9c7\" (UID: \"700242dd-718f-4d0f-9a09-410717e2a0fa\") " pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:33:59.390367 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:59.390241    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/700242dd-718f-4d0f-9a09-410717e2a0fa-kubelet-config\") pod \"global-pull-secret-syncer-cw9c7\" (UID: \"700242dd-718f-4d0f-9a09-410717e2a0fa\") " pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:33:59.390367 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:59.390312    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/700242dd-718f-4d0f-9a09-410717e2a0fa-dbus\") pod \"global-pull-secret-syncer-cw9c7\" (UID: \"700242dd-718f-4d0f-9a09-410717e2a0fa\") " pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:33:59.491458 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:59.491422    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret\") pod \"global-pull-secret-syncer-cw9c7\" (UID: \"700242dd-718f-4d0f-9a09-410717e2a0fa\") " pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:33:59.491620 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:59.491480    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/700242dd-718f-4d0f-9a09-410717e2a0fa-kubelet-config\") pod \"global-pull-secret-syncer-cw9c7\" (UID: \"700242dd-718f-4d0f-9a09-410717e2a0fa\") " pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:33:59.491620 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:59.491508    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/700242dd-718f-4d0f-9a09-410717e2a0fa-dbus\") pod \"global-pull-secret-syncer-cw9c7\" (UID: \"700242dd-718f-4d0f-9a09-410717e2a0fa\") " pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:33:59.491620 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:59.491596    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/700242dd-718f-4d0f-9a09-410717e2a0fa-kubelet-config\") pod \"global-pull-secret-syncer-cw9c7\" (UID: \"700242dd-718f-4d0f-9a09-410717e2a0fa\") " pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:33:59.491830 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:59.491604    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 03 16:33:59.491830 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:59.491655    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/700242dd-718f-4d0f-9a09-410717e2a0fa-dbus\") pod \"global-pull-secret-syncer-cw9c7\" (UID: \"700242dd-718f-4d0f-9a09-410717e2a0fa\") " pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:33:59.491830 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:59.491707    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret podName:700242dd-718f-4d0f-9a09-410717e2a0fa nodeName:}" failed. No retries permitted until 2026-06-03 16:33:59.991687501 +0000 UTC m=+9.522207065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret") pod "global-pull-secret-syncer-cw9c7" (UID: "700242dd-718f-4d0f-9a09-410717e2a0fa") : object "kube-system"/"original-pull-secret" not registered
Jun 03 16:33:59.995891 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:33:59.995839    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret\") pod \"global-pull-secret-syncer-cw9c7\" (UID: \"700242dd-718f-4d0f-9a09-410717e2a0fa\") " pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:33:59.996182 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:59.996009    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 03 16:33:59.996182 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:33:59.996078    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret podName:700242dd-718f-4d0f-9a09-410717e2a0fa nodeName:}" failed. No retries permitted until 2026-06-03 16:34:00.996061303 +0000 UTC m=+10.526580887 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret") pod "global-pull-secret-syncer-cw9c7" (UID: "700242dd-718f-4d0f-9a09-410717e2a0fa") : object "kube-system"/"original-pull-secret" not registered
Jun 03 16:34:00.068657 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:00.068626    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:00.069148 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:00.068773    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:34:00.601277 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:00.601230    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs\") pod \"network-metrics-daemon-q8x4r\" (UID: \"d1292e61-8cac-4672-af45-d434487b3426\") " pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:00.601448 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:00.601367    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 16:34:00.601448 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:00.601446    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs podName:d1292e61-8cac-4672-af45-d434487b3426 nodeName:}" failed. No retries permitted until 2026-06-03 16:34:08.601420321 +0000 UTC m=+18.131939905 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs") pod "network-metrics-daemon-q8x4r" (UID: "d1292e61-8cac-4672-af45-d434487b3426") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 16:34:00.702069 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:00.702034    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qfz5b\" (UniqueName: \"kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b\") pod \"network-check-target-bs44f\" (UID: \"cad8c088-209d-4676-b925-be649bcf3f51\") " pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:00.702240 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:00.702220    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 03 16:34:00.702328 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:00.702248    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 03 16:34:00.702328 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:00.702261    2569 projected.go:194] Error preparing data for projected volume kube-api-access-qfz5b for pod openshift-network-diagnostics/network-check-target-bs44f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 16:34:00.702328 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:00.702325    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b podName:cad8c088-209d-4676-b925-be649bcf3f51 nodeName:}" failed. No retries permitted until 2026-06-03 16:34:08.702307641 +0000 UTC m=+18.232827205 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfz5b" (UniqueName: "kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b") pod "network-check-target-bs44f" (UID: "cad8c088-209d-4676-b925-be649bcf3f51") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 16:34:01.004526 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:01.004446    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret\") pod \"global-pull-secret-syncer-cw9c7\" (UID: \"700242dd-718f-4d0f-9a09-410717e2a0fa\") " pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:01.004689 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:01.004602    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 03 16:34:01.004689 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:01.004673    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret podName:700242dd-718f-4d0f-9a09-410717e2a0fa nodeName:}" failed. No retries permitted until 2026-06-03 16:34:03.004652054 +0000 UTC m=+12.535171637 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret") pod "global-pull-secret-syncer-cw9c7" (UID: "700242dd-718f-4d0f-9a09-410717e2a0fa") : object "kube-system"/"original-pull-secret" not registered
Jun 03 16:34:01.069723 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:01.069691    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:01.070157 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:01.069820    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:01.070157 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:01.069832    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cw9c7" podUID="700242dd-718f-4d0f-9a09-410717e2a0fa"
Jun 03 16:34:01.070157 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:01.069910    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:34:02.068200 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:02.068145    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:02.068400 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:02.068292    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:34:03.022181 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:03.022122    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret\") pod \"global-pull-secret-syncer-cw9c7\" (UID: \"700242dd-718f-4d0f-9a09-410717e2a0fa\") " pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:03.022682 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:03.022305    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 03 16:34:03.022682 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:03.022398    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret podName:700242dd-718f-4d0f-9a09-410717e2a0fa nodeName:}" failed. No retries permitted until 2026-06-03 16:34:07.022375938 +0000 UTC m=+16.552895501 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret") pod "global-pull-secret-syncer-cw9c7" (UID: "700242dd-718f-4d0f-9a09-410717e2a0fa") : object "kube-system"/"original-pull-secret" not registered
Jun 03 16:34:03.068547 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:03.068508    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:03.068725 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:03.068651    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cw9c7" podUID="700242dd-718f-4d0f-9a09-410717e2a0fa"
Jun 03 16:34:03.069053 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:03.069035    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:03.069181 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:03.069136    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:34:04.067940 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:04.067905    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:04.068413 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:04.068043    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:34:04.248955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:04.248715    2569 generic.go:358] "Generic (PLEG): container finished" podID="f8b6b06a-4893-46c5-b030-88a2c01abe88" containerID="6cbec2c903bc70d02f7dfbb546fae4b9ade93af6e7452f8bf34f592a6191fca0" exitCode=0
Jun 03 16:34:04.249137 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:04.248824    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmmz5" event={"ID":"f8b6b06a-4893-46c5-b030-88a2c01abe88","Type":"ContainerDied","Data":"6cbec2c903bc70d02f7dfbb546fae4b9ade93af6e7452f8bf34f592a6191fca0"}
Jun 03 16:34:04.250451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:04.250423    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ccbzr" event={"ID":"c0fd9ccf-e728-4aba-9249-daed366df1e5","Type":"ContainerStarted","Data":"2bd9ebab6f10de21566dc414f239c52d7d01f80a28887b5ce67bf120d07605d8"}
Jun 03 16:34:04.252056 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:04.251980    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xz9hl" event={"ID":"08a04603-3ecf-4148-96c1-21ca6a2a75bf","Type":"ContainerStarted","Data":"0da58a5e7c96bc0f6cfbb11c9721f13e7e505d87ea05ee692ce903f35b222aab"}
Jun 03 16:34:04.253760 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:04.253741    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j5pvb" event={"ID":"5023b960-cc21-4146-bab5-a63abb187a37","Type":"ContainerStarted","Data":"756da5c637e2ef0d204c415436fccc619ae3bb7ba649a522c95b52db1590f823"}
Jun 03 16:34:04.255329 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:04.255297    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj" event={"ID":"0655ed2c-0ff0-498e-af5a-eca49b7663e4","Type":"ContainerStarted","Data":"107793f041345ec9fe722e546c3aabd219beb14bc1a817d848aa097f1487dec0"}
Jun 03 16:34:04.256646 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:04.256619    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vzp5g" event={"ID":"fefcdfd2-e498-45e1-ae6d-fa8431ba97fa","Type":"ContainerStarted","Data":"81b14c712128313e4e4eb5c668c0c831b13526ff2bd0032e3b4da00258a33571"}
Jun 03 16:34:04.281812 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:04.281737    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-j5pvb" podStartSLOduration=3.585126543 podStartE2EDuration="13.281719224s" podCreationTimestamp="2026-06-03 16:33:51 +0000 UTC" firstStartedPulling="2026-06-03 16:33:53.565692769 +0000 UTC m=+3.096212330" lastFinishedPulling="2026-06-03 16:34:03.262285432 +0000 UTC m=+12.792805011" observedRunningTime="2026-06-03 16:34:04.281581203 +0000 UTC m=+13.812100797" watchObservedRunningTime="2026-06-03 16:34:04.281719224 +0000 UTC m=+13.812238811"
Jun 03 16:34:04.309958 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:04.309908    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-vzp5g" podStartSLOduration=3.590485652 podStartE2EDuration="13.309892831s" podCreationTimestamp="2026-06-03 16:33:51 +0000 UTC" firstStartedPulling="2026-06-03 16:33:53.571230143 +0000 UTC m=+3.101749718" lastFinishedPulling="2026-06-03 16:34:03.290637337 +0000 UTC m=+12.821156897" observedRunningTime="2026-06-03 16:34:04.309516568 +0000 UTC m=+13.840036152" watchObservedRunningTime="2026-06-03 16:34:04.309892831 +0000 UTC m=+13.840412417"
Jun 03 16:34:04.310166 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:04.310142    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ccbzr" podStartSLOduration=3.586991942 podStartE2EDuration="13.310135938s" podCreationTimestamp="2026-06-03 16:33:51 +0000 UTC" firstStartedPulling="2026-06-03 16:33:53.56315323 +0000 UTC m=+3.093672797" lastFinishedPulling="2026-06-03 16:34:03.286297222 +0000 UTC m=+12.816816793" observedRunningTime="2026-06-03 16:34:04.293815764 +0000 UTC m=+13.824335341" watchObservedRunningTime="2026-06-03 16:34:04.310135938 +0000 UTC m=+13.840655521"
Jun 03 16:34:04.325673 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:04.325588    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xz9hl" podStartSLOduration=3.5816786 podStartE2EDuration="13.325575619s" podCreationTimestamp="2026-06-03 16:33:51 +0000 UTC" firstStartedPulling="2026-06-03 16:33:53.565861959 +0000 UTC m=+3.096381535" lastFinishedPulling="2026-06-03 16:34:03.309758975 +0000 UTC m=+12.840278554" observedRunningTime="2026-06-03 16:34:04.325089744 +0000 UTC m=+13.855609340" watchObservedRunningTime="2026-06-03 16:34:04.325575619 +0000 UTC m=+13.856095202"
Jun 03 16:34:04.877163 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:04.877130    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-j5pvb"
Jun 03 16:34:04.877763 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:04.877740    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-j5pvb"
Jun 03 16:34:05.068619 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:05.068579    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:05.069061 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:05.068622    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:05.069061 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:05.068727    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:34:05.069061 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:05.068861    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cw9c7" podUID="700242dd-718f-4d0f-9a09-410717e2a0fa"
Jun 03 16:34:05.260066 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:05.260026    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bmrbz" event={"ID":"aab5d542-5157-42c9-9cca-2190dbc377bb","Type":"ContainerStarted","Data":"55b558bff09e699c728df4376f58e538b1479d4973d0b4c3a5ac516350e9fb7d"}
Jun 03 16:34:05.284298 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:05.284252    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bmrbz" podStartSLOduration=4.587806831 podStartE2EDuration="14.284236985s" podCreationTimestamp="2026-06-03 16:33:51 +0000 UTC" firstStartedPulling="2026-06-03 16:33:53.57084047 +0000 UTC m=+3.101360035" lastFinishedPulling="2026-06-03 16:34:03.267270625 +0000 UTC m=+12.797790189" observedRunningTime="2026-06-03 16:34:05.284202781 +0000 UTC m=+14.814722363" watchObservedRunningTime="2026-06-03 16:34:05.284236985 +0000 UTC m=+14.814756566"
Jun 03 16:34:06.068367 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:06.068338    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:06.068523 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:06.068467    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:34:06.261544 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:06.261515    2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jun 03 16:34:07.055768 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:07.055734    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret\") pod \"global-pull-secret-syncer-cw9c7\" (UID: \"700242dd-718f-4d0f-9a09-410717e2a0fa\") " pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:07.055938 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:07.055883    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 03 16:34:07.055986 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:07.055966    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret podName:700242dd-718f-4d0f-9a09-410717e2a0fa nodeName:}" failed. No retries permitted until 2026-06-03 16:34:15.05594945 +0000 UTC m=+24.586469014 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret") pod "global-pull-secret-syncer-cw9c7" (UID: "700242dd-718f-4d0f-9a09-410717e2a0fa") : object "kube-system"/"original-pull-secret" not registered
Jun 03 16:34:07.068058 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:07.068034    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:07.068194 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:07.068146    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:34:07.068251 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:07.068208    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:07.068339 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:07.068319    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cw9c7" podUID="700242dd-718f-4d0f-9a09-410717e2a0fa"
Jun 03 16:34:08.068417 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:08.068368    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:08.069060 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:08.068506    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:34:08.670326 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:08.670287    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs\") pod \"network-metrics-daemon-q8x4r\" (UID: \"d1292e61-8cac-4672-af45-d434487b3426\") " pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:08.670521 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:08.670491    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 16:34:08.670581 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:08.670568    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs podName:d1292e61-8cac-4672-af45-d434487b3426 nodeName:}" failed. No retries permitted until 2026-06-03 16:34:24.670549115 +0000 UTC m=+34.201068680 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs") pod "network-metrics-daemon-q8x4r" (UID: "d1292e61-8cac-4672-af45-d434487b3426") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 16:34:08.770861 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:08.770824    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qfz5b\" (UniqueName: \"kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b\") pod \"network-check-target-bs44f\" (UID: \"cad8c088-209d-4676-b925-be649bcf3f51\") " pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:08.771040 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:08.770995    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 03 16:34:08.771040 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:08.771021    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 03 16:34:08.771040 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:08.771033    2569 projected.go:194] Error preparing data for projected volume kube-api-access-qfz5b for pod openshift-network-diagnostics/network-check-target-bs44f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 16:34:08.771193 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:08.771100    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b podName:cad8c088-209d-4676-b925-be649bcf3f51 nodeName:}" failed. No retries permitted until 2026-06-03 16:34:24.77108062 +0000 UTC m=+34.301600194 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfz5b" (UniqueName: "kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b") pod "network-check-target-bs44f" (UID: "cad8c088-209d-4676-b925-be649bcf3f51") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 16:34:09.068633 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:09.068585    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:09.068633 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:09.068624    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:09.069244 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:09.068731    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:34:09.069244 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:09.068877    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cw9c7" podUID="700242dd-718f-4d0f-9a09-410717e2a0fa"
Jun 03 16:34:09.141883 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:09.141826    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-j5pvb"
Jun 03 16:34:09.142052 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:09.141957    2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jun 03 16:34:09.142630 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:09.142595    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-j5pvb"
Jun 03 16:34:10.068007 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:10.067973    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:10.068190 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:10.068085    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:34:11.069591 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:11.068922    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:11.069591 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:11.069047    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:34:11.069591 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:11.069451    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:11.069591 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:11.069543    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cw9c7" podUID="700242dd-718f-4d0f-9a09-410717e2a0fa"
Jun 03 16:34:12.068448 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:12.068408    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:12.068588 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:12.068505    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:34:13.068131 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:13.068098    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:13.068593 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:13.068111    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:13.068593 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:13.068240    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:34:13.068593 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:13.068296    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cw9c7" podUID="700242dd-718f-4d0f-9a09-410717e2a0fa"
Jun 03 16:34:14.068218 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:14.068063    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:14.068218 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:14.068179    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:34:14.106385 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:14.106357    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jun 03 16:34:14.276522 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:14.276374    2569 generic.go:358] "Generic (PLEG): container finished" podID="f8b6b06a-4893-46c5-b030-88a2c01abe88" containerID="4899abf48dbcedb2ef4b660f8683095297e135d3bfa0415591b5ff77a9299767" exitCode=0
Jun 03 16:34:14.276622 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:14.276444    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmmz5" event={"ID":"f8b6b06a-4893-46c5-b030-88a2c01abe88","Type":"ContainerDied","Data":"4899abf48dbcedb2ef4b660f8683095297e135d3bfa0415591b5ff77a9299767"}
Jun 03 16:34:14.278340 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:14.277897    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s2z68" event={"ID":"a5d651cc-b55b-4806-96ef-1e93043eb8e0","Type":"ContainerStarted","Data":"e4858ba688b7931298b8aef50ee09196b6e607a32fe140b5e2a96fe924bc2541"}
Jun 03 16:34:14.279688 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:14.279664    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj" event={"ID":"0655ed2c-0ff0-498e-af5a-eca49b7663e4","Type":"ContainerStarted","Data":"3f2bb7af6ffd30b23bfb9f95999e733353e22e9b54627069fcceb4343385ec2a"}
Jun 03 16:34:14.283902 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:14.283879    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4" event={"ID":"bbeda011-ea40-4374-b7bf-cb93cae95d56","Type":"ContainerStarted","Data":"e64226a7eb34f40933ec14e230cdc041b94a2e99d3cda8dae889cafcde367761"}
Jun 03 16:34:14.283989 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:14.283908    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4" event={"ID":"bbeda011-ea40-4374-b7bf-cb93cae95d56","Type":"ContainerStarted","Data":"d6e49155af59d83bd6743adbeaea46ec181ccfc7155d3499dd24b2004f1fc0df"}
Jun 03 16:34:14.283989 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:14.283923    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4" event={"ID":"bbeda011-ea40-4374-b7bf-cb93cae95d56","Type":"ContainerStarted","Data":"de5c74633494f062a1c26d457b3c4078bcc47ae0d05fd3f4290fc06373a478ce"}
Jun 03 16:34:14.283989 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:14.283949    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4" event={"ID":"bbeda011-ea40-4374-b7bf-cb93cae95d56","Type":"ContainerStarted","Data":"54e4a1ee98da6c489fa8c37083bcc143c56f9b1c5d0275a9582063213ab66758"}
Jun 03 16:34:14.316407 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:14.316344    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-s2z68" podStartSLOduration=2.900836418 podStartE2EDuration="23.316326855s" podCreationTimestamp="2026-06-03 16:33:51 +0000 UTC" firstStartedPulling="2026-06-03 16:33:53.570014433 +0000 UTC m=+3.100533993" lastFinishedPulling="2026-06-03 16:34:13.98550487 +0000 UTC m=+23.516024430" observedRunningTime="2026-06-03 16:34:14.31546844 +0000 UTC m=+23.845988033" watchObservedRunningTime="2026-06-03 16:34:14.316326855 +0000 UTC m=+23.846846438"
Jun 03 16:34:15.001214 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:15.001115    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-03T16:34:14.106380321Z","UUID":"03cbeea0-ef0f-4c12-a242-db63ed941043","Handler":null,"Name":"","Endpoint":""}
Jun 03 16:34:15.004098 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:15.004074    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jun 03 16:34:15.004098 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:15.004100    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jun 03 16:34:15.069153 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:15.069112    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:15.069560 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:15.069103    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:15.069560 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:15.069306    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cw9c7" podUID="700242dd-718f-4d0f-9a09-410717e2a0fa"
Jun 03 16:34:15.070269 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:15.070239    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:34:15.123544 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:15.123471    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret\") pod \"global-pull-secret-syncer-cw9c7\" (UID: \"700242dd-718f-4d0f-9a09-410717e2a0fa\") " pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:15.123690 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:15.123583    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jun 03 16:34:15.123690 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:15.123654    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret podName:700242dd-718f-4d0f-9a09-410717e2a0fa nodeName:}" failed. No retries permitted until 2026-06-03 16:34:31.123633701 +0000 UTC m=+40.654153263 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret") pod "global-pull-secret-syncer-cw9c7" (UID: "700242dd-718f-4d0f-9a09-410717e2a0fa") : object "kube-system"/"original-pull-secret" not registered
Jun 03 16:34:15.287316 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:15.287282    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj" event={"ID":"0655ed2c-0ff0-498e-af5a-eca49b7663e4","Type":"ContainerStarted","Data":"7b59413d7739fffc09bbe60729f634a1afb24e5390a271c5136612777ad2d024"}
Jun 03 16:34:15.290141 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:15.290114    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4" event={"ID":"bbeda011-ea40-4374-b7bf-cb93cae95d56","Type":"ContainerStarted","Data":"4992e26befb8fb295f1594cbd47b4681452e7aa3f970038f4ce616064a106bc5"}
Jun 03 16:34:15.290235 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:15.290151    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4" event={"ID":"bbeda011-ea40-4374-b7bf-cb93cae95d56","Type":"ContainerStarted","Data":"891053ba8887ae9b60437e6b877247ed27c09f634fb7bd94added0b33db0a156"}
Jun 03 16:34:15.291962 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:15.291937    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmmz5" event={"ID":"f8b6b06a-4893-46c5-b030-88a2c01abe88","Type":"ContainerStarted","Data":"ac8f562a7fa33af6d8681512b5935cb463348c54004daae8793fe78894db9960"}
Jun 03 16:34:15.305747 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:15.305705    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9pgpj" podStartSLOduration=3.022855498 podStartE2EDuration="24.305691005s" podCreationTimestamp="2026-06-03 16:33:51 +0000 UTC" firstStartedPulling="2026-06-03 16:33:53.573376522 +0000 UTC m=+3.103896097" lastFinishedPulling="2026-06-03 16:34:14.856212039 +0000 UTC m=+24.386731604" observedRunningTime="2026-06-03 16:34:15.305123802 +0000 UTC m=+24.835643384" watchObservedRunningTime="2026-06-03 16:34:15.305691005 +0000 UTC m=+24.836210589"
Jun 03 16:34:16.067861 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:16.067827    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:16.068037 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:16.067929    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:34:16.294854 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:16.294820    2569 generic.go:358] "Generic (PLEG): container finished" podID="f8b6b06a-4893-46c5-b030-88a2c01abe88" containerID="ac8f562a7fa33af6d8681512b5935cb463348c54004daae8793fe78894db9960" exitCode=0
Jun 03 16:34:16.295308 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:16.294894    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmmz5" event={"ID":"f8b6b06a-4893-46c5-b030-88a2c01abe88","Type":"ContainerDied","Data":"ac8f562a7fa33af6d8681512b5935cb463348c54004daae8793fe78894db9960"}
Jun 03 16:34:17.068444 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:17.068411    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:17.068444 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:17.068429    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:17.068678 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:17.068523    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:34:17.068678 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:17.068658    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cw9c7" podUID="700242dd-718f-4d0f-9a09-410717e2a0fa"
Jun 03 16:34:17.299387 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:17.299358    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4" event={"ID":"bbeda011-ea40-4374-b7bf-cb93cae95d56","Type":"ContainerStarted","Data":"be9e61a98138fa93cacffac7ee39d064ee717cba6eae8661e0752a5736e14313"}
Jun 03 16:34:18.068504 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:18.068465    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:18.068698 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:18.068573    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:34:18.302758 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:18.302726    2569 generic.go:358] "Generic (PLEG): container finished" podID="f8b6b06a-4893-46c5-b030-88a2c01abe88" containerID="133fff961929092c78bbf61f9e326e0fe7256eb300b88a9640724dc8cb276317" exitCode=0
Jun 03 16:34:18.303149 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:18.302769    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmmz5" event={"ID":"f8b6b06a-4893-46c5-b030-88a2c01abe88","Type":"ContainerDied","Data":"133fff961929092c78bbf61f9e326e0fe7256eb300b88a9640724dc8cb276317"}
Jun 03 16:34:19.068517 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:19.068247    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:19.068692 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:19.068317    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:19.068692 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:19.068629    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cw9c7" podUID="700242dd-718f-4d0f-9a09-410717e2a0fa"
Jun 03 16:34:19.068841 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:19.068728    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:34:19.308535 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:19.308499    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4" event={"ID":"bbeda011-ea40-4374-b7bf-cb93cae95d56","Type":"ContainerStarted","Data":"262111a1a4707eeff45e794455eee6c53ba7e1eb1df64f57d9cd8341f0c9059a"}
Jun 03 16:34:19.309186 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:19.308850    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:34:19.309186 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:19.308874    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:34:19.326026 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:19.325965    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:34:19.335927 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:19.335887    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4" podStartSLOduration=7.921469161 podStartE2EDuration="28.335874048s" podCreationTimestamp="2026-06-03 16:33:51 +0000 UTC" firstStartedPulling="2026-06-03 16:33:53.568821786 +0000 UTC m=+3.099341363" lastFinishedPulling="2026-06-03 16:34:13.983226685 +0000 UTC m=+23.513746250" observedRunningTime="2026-06-03 16:34:19.335554773 +0000 UTC m=+28.866074356" watchObservedRunningTime="2026-06-03 16:34:19.335874048 +0000 UTC m=+28.866393630"
Jun 03 16:34:20.068355 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:20.068320    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:20.068576 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:20.068448    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:34:20.311517 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:20.311476    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:34:20.329528 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:20.329466    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:34:20.669125 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:20.669047    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bs44f"]
Jun 03 16:34:20.669329 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:20.669139    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:20.669329 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:20.669239    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:34:20.672861 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:20.672837    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cw9c7"]
Jun 03 16:34:20.672976 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:20.672937    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:20.673038 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:20.673021    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cw9c7" podUID="700242dd-718f-4d0f-9a09-410717e2a0fa"
Jun 03 16:34:20.673511 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:20.673490    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-q8x4r"]
Jun 03 16:34:20.673624 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:20.673597    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:20.673736 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:20.673699    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:34:22.068114 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:22.068081    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:22.068521 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:22.068081    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:22.068521 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:22.068194    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:34:22.068521 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:22.068081    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:22.068521 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:22.068267    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cw9c7" podUID="700242dd-718f-4d0f-9a09-410717e2a0fa"
Jun 03 16:34:22.068521 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:22.068328    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:34:24.068734 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:24.068701    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:24.068734 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:24.068727    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:24.068734 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:24.068743    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:24.069170 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:24.068853    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:34:24.069170 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:24.068960    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:34:24.069170 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:24.069017    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cw9c7" podUID="700242dd-718f-4d0f-9a09-410717e2a0fa"
Jun 03 16:34:24.700962 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:24.700929    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs\") pod \"network-metrics-daemon-q8x4r\" (UID: \"d1292e61-8cac-4672-af45-d434487b3426\") " pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:24.701121 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:24.701071    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 16:34:24.701167 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:24.701135    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs podName:d1292e61-8cac-4672-af45-d434487b3426 nodeName:}" failed. No retries permitted until 2026-06-03 16:34:56.701118475 +0000 UTC m=+66.231638040 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs") pod "network-metrics-daemon-q8x4r" (UID: "d1292e61-8cac-4672-af45-d434487b3426") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 03 16:34:24.801413 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:24.801381    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qfz5b\" (UniqueName: \"kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b\") pod \"network-check-target-bs44f\" (UID: \"cad8c088-209d-4676-b925-be649bcf3f51\") " pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:24.801552 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:24.801525    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 03 16:34:24.801552 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:24.801542    2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 03 16:34:24.801552 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:24.801552    2569 projected.go:194] Error preparing data for projected volume kube-api-access-qfz5b for pod openshift-network-diagnostics/network-check-target-bs44f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 16:34:24.801682 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:24.801605    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b podName:cad8c088-209d-4676-b925-be649bcf3f51 nodeName:}" failed. No retries permitted until 2026-06-03 16:34:56.801586469 +0000 UTC m=+66.332106035 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qfz5b" (UniqueName: "kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b") pod "network-check-target-bs44f" (UID: "cad8c088-209d-4676-b925-be649bcf3f51") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 03 16:34:26.068141 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:26.068105    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:26.068614 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:26.068105    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:26.068614 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:26.068210    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs44f" podUID="cad8c088-209d-4676-b925-be649bcf3f51"
Jun 03 16:34:26.068614 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:26.068250    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:26.068614 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:26.068318    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cw9c7" podUID="700242dd-718f-4d0f-9a09-410717e2a0fa"
Jun 03 16:34:26.068614 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:26.068418    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:34:27.778614 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:27.778443    2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-123.ec2.internal" event="NodeReady"
Jun 03 16:34:27.779190 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:27.778734    2569 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
Jun 03 16:34:27.830137 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:27.830066    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-x7fvf"]
Jun 03 16:34:27.841487 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:27.841461    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tdqmj"]
Jun 03 16:34:27.841646 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:27.841629    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:27.844449 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:27.844425    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6d9kw\""
Jun 03 16:34:27.844449 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:27.844430    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jun 03 16:34:27.844954 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:27.844932    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jun 03 16:34:27.856635 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:27.856615    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x7fvf"]
Jun 03 16:34:27.856635 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:27.856637    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tdqmj"]
Jun 03 16:34:27.856758 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:27.856734    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:34:27.859581 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:27.859559    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jun 03 16:34:27.859689 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:27.859613    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jun 03 16:34:27.859689 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:27.859660    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jun 03 16:34:27.859809 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:27.859700    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kldw8\""
Jun 03 16:34:28.024813 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.024751    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:28.024986 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.024823    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert\") pod \"ingress-canary-tdqmj\" (UID: \"bd8eaa6c-3029-4458-9d0c-13933d35c3c6\") " pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:34:28.024986 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.024857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-tmp-dir\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:28.024986 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.024884    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-config-volume\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:28.024986 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.024943    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsxkt\" (UniqueName: \"kubernetes.io/projected/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-kube-api-access-hsxkt\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:28.024986 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.024968    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8s6c\" (UniqueName: \"kubernetes.io/projected/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-kube-api-access-c8s6c\") pod \"ingress-canary-tdqmj\" (UID: \"bd8eaa6c-3029-4458-9d0c-13933d35c3c6\") " pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:34:28.068868 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.068836    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:28.068868 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.068854    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:28.069085 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.069049    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:28.071736 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.071715    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jun 03 16:34:28.071907 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.071753    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jun 03 16:34:28.071907 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.071760    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r657c\""
Jun 03 16:34:28.071907 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.071874    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jun 03 16:34:28.072018 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.071989    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jun 03 16:34:28.072018 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.071998    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fx9bm\""
Jun 03 16:34:28.125421 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.125392    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-config-volume\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:28.125597 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.125430    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hsxkt\" (UniqueName: \"kubernetes.io/projected/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-kube-api-access-hsxkt\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:28.125597 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.125448    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c8s6c\" (UniqueName: \"kubernetes.io/projected/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-kube-api-access-c8s6c\") pod \"ingress-canary-tdqmj\" (UID: \"bd8eaa6c-3029-4458-9d0c-13933d35c3c6\") " pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:34:28.125715 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.125601    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:28.125715 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:28.125690    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 16:34:28.125840 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:28.125737    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls podName:9eb09939-3056-4ca9-8d22-64cd87cf4f8e nodeName:}" failed. No retries permitted until 2026-06-03 16:34:28.625723929 +0000 UTC m=+38.156243491 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls") pod "dns-default-x7fvf" (UID: "9eb09939-3056-4ca9-8d22-64cd87cf4f8e") : secret "dns-default-metrics-tls" not found
Jun 03 16:34:28.125840 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.125833    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert\") pod \"ingress-canary-tdqmj\" (UID: \"bd8eaa6c-3029-4458-9d0c-13933d35c3c6\") " pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:34:28.125954 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.125866    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-tmp-dir\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:28.125954 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:28.125912    2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 03 16:34:28.125954 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:28.125948    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert podName:bd8eaa6c-3029-4458-9d0c-13933d35c3c6 nodeName:}" failed. No retries permitted until 2026-06-03 16:34:28.625936409 +0000 UTC m=+38.156455974 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert") pod "ingress-canary-tdqmj" (UID: "bd8eaa6c-3029-4458-9d0c-13933d35c3c6") : secret "canary-serving-cert" not found
Jun 03 16:34:28.126123 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.126106    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-tmp-dir\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:28.126173 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.126106    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-config-volume\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:28.135519 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.135500    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsxkt\" (UniqueName: \"kubernetes.io/projected/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-kube-api-access-hsxkt\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:28.135666 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.135649    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8s6c\" (UniqueName: \"kubernetes.io/projected/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-kube-api-access-c8s6c\") pod \"ingress-canary-tdqmj\" (UID: \"bd8eaa6c-3029-4458-9d0c-13933d35c3c6\") " pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:34:28.327958 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.327922    2569 generic.go:358] "Generic (PLEG): container finished" podID="f8b6b06a-4893-46c5-b030-88a2c01abe88" containerID="ecfb6a36d8884862c61f134a55a809f4e7ab2806af5ddb4bd17f6136425a81c9" exitCode=0
Jun 03 16:34:28.328111 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.327979    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmmz5" event={"ID":"f8b6b06a-4893-46c5-b030-88a2c01abe88","Type":"ContainerDied","Data":"ecfb6a36d8884862c61f134a55a809f4e7ab2806af5ddb4bd17f6136425a81c9"}
Jun 03 16:34:28.629883 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.629776    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:28.629883 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:28.629843    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert\") pod \"ingress-canary-tdqmj\" (UID: \"bd8eaa6c-3029-4458-9d0c-13933d35c3c6\") " pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:34:28.630057 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:28.629933    2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 03 16:34:28.630057 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:28.629937    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 16:34:28.630057 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:28.629989    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert podName:bd8eaa6c-3029-4458-9d0c-13933d35c3c6 nodeName:}" failed. No retries permitted until 2026-06-03 16:34:29.629974409 +0000 UTC m=+39.160493970 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert") pod "ingress-canary-tdqmj" (UID: "bd8eaa6c-3029-4458-9d0c-13933d35c3c6") : secret "canary-serving-cert" not found
Jun 03 16:34:28.630057 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:28.630002    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls podName:9eb09939-3056-4ca9-8d22-64cd87cf4f8e nodeName:}" failed. No retries permitted until 2026-06-03 16:34:29.629996563 +0000 UTC m=+39.160516123 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls") pod "dns-default-x7fvf" (UID: "9eb09939-3056-4ca9-8d22-64cd87cf4f8e") : secret "dns-default-metrics-tls" not found
Jun 03 16:34:29.332386 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:29.332348    2569 generic.go:358] "Generic (PLEG): container finished" podID="f8b6b06a-4893-46c5-b030-88a2c01abe88" containerID="ebde522562b1620930db850caea062663b955d890d3c21bb0a2dbf4b2a007899" exitCode=0
Jun 03 16:34:29.332885 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:29.332403    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmmz5" event={"ID":"f8b6b06a-4893-46c5-b030-88a2c01abe88","Type":"ContainerDied","Data":"ebde522562b1620930db850caea062663b955d890d3c21bb0a2dbf4b2a007899"}
Jun 03 16:34:29.637962 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:29.637871    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:29.637962 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:29.637919    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert\") pod \"ingress-canary-tdqmj\" (UID: \"bd8eaa6c-3029-4458-9d0c-13933d35c3c6\") " pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:34:29.638157 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:29.638016    2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 03 16:34:29.638157 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:29.638023    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 16:34:29.638157 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:29.638066    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert podName:bd8eaa6c-3029-4458-9d0c-13933d35c3c6 nodeName:}" failed. No retries permitted until 2026-06-03 16:34:31.638052408 +0000 UTC m=+41.168571969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert") pod "ingress-canary-tdqmj" (UID: "bd8eaa6c-3029-4458-9d0c-13933d35c3c6") : secret "canary-serving-cert" not found
Jun 03 16:34:29.638157 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:29.638080    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls podName:9eb09939-3056-4ca9-8d22-64cd87cf4f8e nodeName:}" failed. No retries permitted until 2026-06-03 16:34:31.638074062 +0000 UTC m=+41.168593623 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls") pod "dns-default-x7fvf" (UID: "9eb09939-3056-4ca9-8d22-64cd87cf4f8e") : secret "dns-default-metrics-tls" not found
Jun 03 16:34:30.336971 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:30.336941    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmmz5" event={"ID":"f8b6b06a-4893-46c5-b030-88a2c01abe88","Type":"ContainerStarted","Data":"6b57798f3eaa3c080e81126b1b6027949c0095efcb24f3a68611b4cca43c4396"}
Jun 03 16:34:30.360010 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:30.359968    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qmmz5" podStartSLOduration=5.628616028 podStartE2EDuration="39.359955238s" podCreationTimestamp="2026-06-03 16:33:51 +0000 UTC" firstStartedPulling="2026-06-03 16:33:53.564324679 +0000 UTC m=+3.094844240" lastFinishedPulling="2026-06-03 16:34:27.295663888 +0000 UTC m=+36.826183450" observedRunningTime="2026-06-03 16:34:30.358007195 +0000 UTC m=+39.888526778" watchObservedRunningTime="2026-06-03 16:34:30.359955238 +0000 UTC m=+39.890474852"
Jun 03 16:34:31.149326 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.149279    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret\") pod \"global-pull-secret-syncer-cw9c7\" (UID: \"700242dd-718f-4d0f-9a09-410717e2a0fa\") " pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:31.151653 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.151635    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/700242dd-718f-4d0f-9a09-410717e2a0fa-original-pull-secret\") pod \"global-pull-secret-syncer-cw9c7\" (UID: \"700242dd-718f-4d0f-9a09-410717e2a0fa\") " pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:31.293757 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.293724    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76cf9dff44-d8hkh"]
Jun 03 16:34:31.298020 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.297999    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76cf9dff44-d8hkh"
Jun 03 16:34:31.300929 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.300902    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jun 03 16:34:31.300929 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.300919    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jun 03 16:34:31.301106 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.300912    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jun 03 16:34:31.302439 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.302342    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jun 03 16:34:31.302534 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.302344    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-bmh9g\""
Jun 03 16:34:31.303286 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.303265    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76cf9dff44-d8hkh"]
Jun 03 16:34:31.306916 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.306898    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"]
Jun 03 16:34:31.310371 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.310354    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"]
Jun 03 16:34:31.310541 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.310523    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"
Jun 03 16:34:31.313224 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.313207    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jun 03 16:34:31.313911 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.313898    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.316576 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.316557    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jun 03 16:34:31.316654 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.316575    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jun 03 16:34:31.316697 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.316654    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jun 03 16:34:31.316739 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.316727    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jun 03 16:34:31.318938 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.318921    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"]
Jun 03 16:34:31.320248 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.320227    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"]
Jun 03 16:34:31.388744 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.388716    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cw9c7"
Jun 03 16:34:31.452001 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.451822    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6b7e03c1-b306-4d27-8ea5-959a63034a79-hub\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.452143 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.452035    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/6b7e03c1-b306-4d27-8ea5-959a63034a79-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.452143 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.452072    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6b7e03c1-b306-4d27-8ea5-959a63034a79-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.452215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.452138    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6089ff9d-c8ed-4474-a288-c01b9f7baca9-tmp\") pod \"klusterlet-addon-workmgr-cdf67d4b5-wqcvr\" (UID: \"6089ff9d-c8ed-4474-a288-c01b9f7baca9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"
Jun 03 16:34:31.452215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.452180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/071a20f8-777a-4d61-b6bd-b35e61799652-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-76cf9dff44-d8hkh\" (UID: \"071a20f8-777a-4d61-b6bd-b35e61799652\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76cf9dff44-d8hkh"
Jun 03 16:34:31.452215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.452208    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6b7e03c1-b306-4d27-8ea5-959a63034a79-ca\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.452312 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.452233    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5sh6\" (UniqueName: \"kubernetes.io/projected/6089ff9d-c8ed-4474-a288-c01b9f7baca9-kube-api-access-b5sh6\") pod \"klusterlet-addon-workmgr-cdf67d4b5-wqcvr\" (UID: \"6089ff9d-c8ed-4474-a288-c01b9f7baca9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"
Jun 03 16:34:31.452346 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.452310    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6089ff9d-c8ed-4474-a288-c01b9f7baca9-klusterlet-config\") pod \"klusterlet-addon-workmgr-cdf67d4b5-wqcvr\" (UID: \"6089ff9d-c8ed-4474-a288-c01b9f7baca9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"
Jun 03 16:34:31.452409 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.452396    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s57gk\" (UniqueName: \"kubernetes.io/projected/071a20f8-777a-4d61-b6bd-b35e61799652-kube-api-access-s57gk\") pod \"managed-serviceaccount-addon-agent-76cf9dff44-d8hkh\" (UID: \"071a20f8-777a-4d61-b6bd-b35e61799652\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76cf9dff44-d8hkh"
Jun 03 16:34:31.452460 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.452416    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b7e03c1-b306-4d27-8ea5-959a63034a79-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.452510 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.452472    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x9v7\" (UniqueName: \"kubernetes.io/projected/6b7e03c1-b306-4d27-8ea5-959a63034a79-kube-api-access-5x9v7\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.513060 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.513034    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cw9c7"]
Jun 03 16:34:31.516726 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:34:31.516693    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod700242dd_718f_4d0f_9a09_410717e2a0fa.slice/crio-bbcfb2f0e9fe6eb21a5a222a8eb75d3adeff4914d0a5ec243c91810915ec5934 WatchSource:0}: Error finding container bbcfb2f0e9fe6eb21a5a222a8eb75d3adeff4914d0a5ec243c91810915ec5934: Status 404 returned error can't find the container with id bbcfb2f0e9fe6eb21a5a222a8eb75d3adeff4914d0a5ec243c91810915ec5934
Jun 03 16:34:31.552992 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.552969    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5x9v7\" (UniqueName: \"kubernetes.io/projected/6b7e03c1-b306-4d27-8ea5-959a63034a79-kube-api-access-5x9v7\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.553107 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.553023    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6b7e03c1-b306-4d27-8ea5-959a63034a79-hub\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.553159 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.553136    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/6b7e03c1-b306-4d27-8ea5-959a63034a79-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.553211 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.553167    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6b7e03c1-b306-4d27-8ea5-959a63034a79-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.553211 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.553197    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6089ff9d-c8ed-4474-a288-c01b9f7baca9-tmp\") pod \"klusterlet-addon-workmgr-cdf67d4b5-wqcvr\" (UID: \"6089ff9d-c8ed-4474-a288-c01b9f7baca9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"
Jun 03 16:34:31.553347 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.553271    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/071a20f8-777a-4d61-b6bd-b35e61799652-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-76cf9dff44-d8hkh\" (UID: \"071a20f8-777a-4d61-b6bd-b35e61799652\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76cf9dff44-d8hkh"
Jun 03 16:34:31.553347 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.553309    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6b7e03c1-b306-4d27-8ea5-959a63034a79-ca\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.553347 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.553346    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b5sh6\" (UniqueName: \"kubernetes.io/projected/6089ff9d-c8ed-4474-a288-c01b9f7baca9-kube-api-access-b5sh6\") pod \"klusterlet-addon-workmgr-cdf67d4b5-wqcvr\" (UID: \"6089ff9d-c8ed-4474-a288-c01b9f7baca9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"
Jun 03 16:34:31.553508 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.553395    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6089ff9d-c8ed-4474-a288-c01b9f7baca9-klusterlet-config\") pod \"klusterlet-addon-workmgr-cdf67d4b5-wqcvr\" (UID: \"6089ff9d-c8ed-4474-a288-c01b9f7baca9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"
Jun 03 16:34:31.553508 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.553440    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s57gk\" (UniqueName: \"kubernetes.io/projected/071a20f8-777a-4d61-b6bd-b35e61799652-kube-api-access-s57gk\") pod \"managed-serviceaccount-addon-agent-76cf9dff44-d8hkh\" (UID: \"071a20f8-777a-4d61-b6bd-b35e61799652\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76cf9dff44-d8hkh"
Jun 03 16:34:31.553508 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.553469    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b7e03c1-b306-4d27-8ea5-959a63034a79-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.553710 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.553687    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6089ff9d-c8ed-4474-a288-c01b9f7baca9-tmp\") pod \"klusterlet-addon-workmgr-cdf67d4b5-wqcvr\" (UID: \"6089ff9d-c8ed-4474-a288-c01b9f7baca9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"
Jun 03 16:34:31.554002 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.553964    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/6b7e03c1-b306-4d27-8ea5-959a63034a79-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.556525 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.556501    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6b7e03c1-b306-4d27-8ea5-959a63034a79-ca\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.556712 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.556689    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/071a20f8-777a-4d61-b6bd-b35e61799652-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-76cf9dff44-d8hkh\" (UID: \"071a20f8-777a-4d61-b6bd-b35e61799652\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76cf9dff44-d8hkh"
Jun 03 16:34:31.556877 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.556858    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6b7e03c1-b306-4d27-8ea5-959a63034a79-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.556959 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.556931    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b7e03c1-b306-4d27-8ea5-959a63034a79-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.557037 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.557017    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6b7e03c1-b306-4d27-8ea5-959a63034a79-hub\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.557100 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.557065    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6089ff9d-c8ed-4474-a288-c01b9f7baca9-klusterlet-config\") pod \"klusterlet-addon-workmgr-cdf67d4b5-wqcvr\" (UID: \"6089ff9d-c8ed-4474-a288-c01b9f7baca9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"
Jun 03 16:34:31.561442 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.561414    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x9v7\" (UniqueName: \"kubernetes.io/projected/6b7e03c1-b306-4d27-8ea5-959a63034a79-kube-api-access-5x9v7\") pod \"cluster-proxy-proxy-agent-57cd466ff-tzvvz\" (UID: \"6b7e03c1-b306-4d27-8ea5-959a63034a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.561597 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.561577    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s57gk\" (UniqueName: \"kubernetes.io/projected/071a20f8-777a-4d61-b6bd-b35e61799652-kube-api-access-s57gk\") pod \"managed-serviceaccount-addon-agent-76cf9dff44-d8hkh\" (UID: \"071a20f8-777a-4d61-b6bd-b35e61799652\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76cf9dff44-d8hkh"
Jun 03 16:34:31.561724 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.561705    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5sh6\" (UniqueName: \"kubernetes.io/projected/6089ff9d-c8ed-4474-a288-c01b9f7baca9-kube-api-access-b5sh6\") pod \"klusterlet-addon-workmgr-cdf67d4b5-wqcvr\" (UID: \"6089ff9d-c8ed-4474-a288-c01b9f7baca9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"
Jun 03 16:34:31.621311 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.621279    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76cf9dff44-d8hkh"
Jun 03 16:34:31.630089 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.630068    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"
Jun 03 16:34:31.635752 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.635729    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:34:31.654696 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.654668    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:31.654872 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.654704    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert\") pod \"ingress-canary-tdqmj\" (UID: \"bd8eaa6c-3029-4458-9d0c-13933d35c3c6\") " pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:34:31.654872 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:31.654861    2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 03 16:34:31.654988 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:31.654912    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert podName:bd8eaa6c-3029-4458-9d0c-13933d35c3c6 nodeName:}" failed. No retries permitted until 2026-06-03 16:34:35.654897012 +0000 UTC m=+45.185416572 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert") pod "ingress-canary-tdqmj" (UID: "bd8eaa6c-3029-4458-9d0c-13933d35c3c6") : secret "canary-serving-cert" not found
Jun 03 16:34:31.655205 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:31.655176    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 16:34:31.655314 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:31.655262    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls podName:9eb09939-3056-4ca9-8d22-64cd87cf4f8e nodeName:}" failed. No retries permitted until 2026-06-03 16:34:35.655242977 +0000 UTC m=+45.185762558 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls") pod "dns-default-x7fvf" (UID: "9eb09939-3056-4ca9-8d22-64cd87cf4f8e") : secret "dns-default-metrics-tls" not found
Jun 03 16:34:31.753837 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.753769    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76cf9dff44-d8hkh"]
Jun 03 16:34:31.757535 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:34:31.757509    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod071a20f8_777a_4d61_b6bd_b35e61799652.slice/crio-28aec8f6a0d85aec3fec99c919f13f2dc25a56d8eabea989eb8db6fe49f9bb43 WatchSource:0}: Error finding container 28aec8f6a0d85aec3fec99c919f13f2dc25a56d8eabea989eb8db6fe49f9bb43: Status 404 returned error can't find the container with id 28aec8f6a0d85aec3fec99c919f13f2dc25a56d8eabea989eb8db6fe49f9bb43
Jun 03 16:34:31.762741 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.762719    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"]
Jun 03 16:34:31.765889 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:34:31.765868    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6089ff9d_c8ed_4474_a288_c01b9f7baca9.slice/crio-599bc822220fd83ebffff84d060e87aae2683c0d7686040bd74ad5b0bf3d16ab WatchSource:0}: Error finding container 599bc822220fd83ebffff84d060e87aae2683c0d7686040bd74ad5b0bf3d16ab: Status 404 returned error can't find the container with id 599bc822220fd83ebffff84d060e87aae2683c0d7686040bd74ad5b0bf3d16ab
Jun 03 16:34:31.775077 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:31.775057    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"]
Jun 03 16:34:31.777176 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:34:31.777151    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b7e03c1_b306_4d27_8ea5_959a63034a79.slice/crio-63ca433ed6f1f42236714c8a3f3dd6244b553531772912c6292cf2c90ccbf3dd WatchSource:0}: Error finding container 63ca433ed6f1f42236714c8a3f3dd6244b553531772912c6292cf2c90ccbf3dd: Status 404 returned error can't find the container with id 63ca433ed6f1f42236714c8a3f3dd6244b553531772912c6292cf2c90ccbf3dd
Jun 03 16:34:32.341403 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:32.341352    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz" event={"ID":"6b7e03c1-b306-4d27-8ea5-959a63034a79","Type":"ContainerStarted","Data":"63ca433ed6f1f42236714c8a3f3dd6244b553531772912c6292cf2c90ccbf3dd"}
Jun 03 16:34:32.342735 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:32.342705    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr" event={"ID":"6089ff9d-c8ed-4474-a288-c01b9f7baca9","Type":"ContainerStarted","Data":"599bc822220fd83ebffff84d060e87aae2683c0d7686040bd74ad5b0bf3d16ab"}
Jun 03 16:34:32.344754 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:32.344705    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76cf9dff44-d8hkh" event={"ID":"071a20f8-777a-4d61-b6bd-b35e61799652","Type":"ContainerStarted","Data":"28aec8f6a0d85aec3fec99c919f13f2dc25a56d8eabea989eb8db6fe49f9bb43"}
Jun 03 16:34:32.345971 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:32.345948    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cw9c7" event={"ID":"700242dd-718f-4d0f-9a09-410717e2a0fa","Type":"ContainerStarted","Data":"bbcfb2f0e9fe6eb21a5a222a8eb75d3adeff4914d0a5ec243c91810915ec5934"}
Jun 03 16:34:35.684734 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:35.684137    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:35.684734 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:35.684447    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert\") pod \"ingress-canary-tdqmj\" (UID: \"bd8eaa6c-3029-4458-9d0c-13933d35c3c6\") " pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:34:35.684734 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:35.684297    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 16:34:35.684734 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:35.684670    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls podName:9eb09939-3056-4ca9-8d22-64cd87cf4f8e nodeName:}" failed. No retries permitted until 2026-06-03 16:34:43.684650514 +0000 UTC m=+53.215170077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls") pod "dns-default-x7fvf" (UID: "9eb09939-3056-4ca9-8d22-64cd87cf4f8e") : secret "dns-default-metrics-tls" not found
Jun 03 16:34:35.685422 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:35.684605    2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 03 16:34:35.685422 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:35.685093    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert podName:bd8eaa6c-3029-4458-9d0c-13933d35c3c6 nodeName:}" failed. No retries permitted until 2026-06-03 16:34:43.685071378 +0000 UTC m=+53.215590943 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert") pod "ingress-canary-tdqmj" (UID: "bd8eaa6c-3029-4458-9d0c-13933d35c3c6") : secret "canary-serving-cert" not found
Jun 03 16:34:39.365808 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:39.365749    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz" event={"ID":"6b7e03c1-b306-4d27-8ea5-959a63034a79","Type":"ContainerStarted","Data":"bc275370db301837dae389342d4cd1617ccc6f4ae730ef8afc46a9734750d1c1"}
Jun 03 16:34:39.367401 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:39.367355    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr" event={"ID":"6089ff9d-c8ed-4474-a288-c01b9f7baca9","Type":"ContainerStarted","Data":"602d5b2db8ff49656f28de4e125a4fa660f318eca7ac387f1ba876db3e758c46"}
Jun 03 16:34:39.367691 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:39.367658    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"
Jun 03 16:34:39.369608 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:39.369583    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"
Jun 03 16:34:39.369782 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:39.369755    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76cf9dff44-d8hkh" event={"ID":"071a20f8-777a-4d61-b6bd-b35e61799652","Type":"ContainerStarted","Data":"621547a172832f230736527834619fca5a9c5b000e88550004bbce8a8e8ba79e"}
Jun 03 16:34:39.383774 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:39.383725    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr" podStartSLOduration=1.842566697 podStartE2EDuration="8.383714128s" podCreationTimestamp="2026-06-03 16:34:31 +0000 UTC" firstStartedPulling="2026-06-03 16:34:31.768990023 +0000 UTC m=+41.299509586" lastFinishedPulling="2026-06-03 16:34:38.310137455 +0000 UTC m=+47.840657017" observedRunningTime="2026-06-03 16:34:39.383050383 +0000 UTC m=+48.913569967" watchObservedRunningTime="2026-06-03 16:34:39.383714128 +0000 UTC m=+48.914233718"
Jun 03 16:34:41.097437 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:41.097378    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76cf9dff44-d8hkh" podStartSLOduration=3.576409822 podStartE2EDuration="10.097359124s" podCreationTimestamp="2026-06-03 16:34:31 +0000 UTC" firstStartedPulling="2026-06-03 16:34:31.759346404 +0000 UTC m=+41.289865965" lastFinishedPulling="2026-06-03 16:34:38.280295701 +0000 UTC m=+47.810815267" observedRunningTime="2026-06-03 16:34:39.413176857 +0000 UTC m=+48.943696440" watchObservedRunningTime="2026-06-03 16:34:41.097359124 +0000 UTC m=+50.627878708"
Jun 03 16:34:42.381064 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:42.380970    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cw9c7" event={"ID":"700242dd-718f-4d0f-9a09-410717e2a0fa","Type":"ContainerStarted","Data":"29700f0a8a516d16db8d96447e2581741a9dbf451044af23d36342c80034318f"}
Jun 03 16:34:42.382904 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:42.382870    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz" event={"ID":"6b7e03c1-b306-4d27-8ea5-959a63034a79","Type":"ContainerStarted","Data":"9389fc237ab53f668207a76f9554b6cc73baa09a9402786f1ece6947a3b94085"}
Jun 03 16:34:42.383031 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:42.382912    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz" event={"ID":"6b7e03c1-b306-4d27-8ea5-959a63034a79","Type":"ContainerStarted","Data":"dae12fa92bd453a0ae68aa3b17810d00af2980debec23ee62b44f8ca4a91e59b"}
Jun 03 16:34:42.394932 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:42.394885    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cw9c7" podStartSLOduration=33.099216155 podStartE2EDuration="43.394869961s" podCreationTimestamp="2026-06-03 16:33:59 +0000 UTC" firstStartedPulling="2026-06-03 16:34:31.518636883 +0000 UTC m=+41.049156444" lastFinishedPulling="2026-06-03 16:34:41.814290674 +0000 UTC m=+51.344810250" observedRunningTime="2026-06-03 16:34:42.394566025 +0000 UTC m=+51.925085607" watchObservedRunningTime="2026-06-03 16:34:42.394869961 +0000 UTC m=+51.925389544"
Jun 03 16:34:42.411576 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:42.411532    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz" podStartSLOduration=1.371626599 podStartE2EDuration="11.411520279s" podCreationTimestamp="2026-06-03 16:34:31 +0000 UTC" firstStartedPulling="2026-06-03 16:34:31.778641661 +0000 UTC m=+41.309161223" lastFinishedPulling="2026-06-03 16:34:41.818535342 +0000 UTC m=+51.349054903" observedRunningTime="2026-06-03 16:34:42.410652133 +0000 UTC m=+51.941171729" watchObservedRunningTime="2026-06-03 16:34:42.411520279 +0000 UTC m=+51.942039856"
Jun 03 16:34:43.745631 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:43.745593    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:43.745631 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:43.745631    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert\") pod \"ingress-canary-tdqmj\" (UID: \"bd8eaa6c-3029-4458-9d0c-13933d35c3c6\") " pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:34:43.746077 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:43.745744    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 16:34:43.746077 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:43.745825    2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 03 16:34:43.746077 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:43.745828    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls podName:9eb09939-3056-4ca9-8d22-64cd87cf4f8e nodeName:}" failed. No retries permitted until 2026-06-03 16:34:59.745812412 +0000 UTC m=+69.276331986 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls") pod "dns-default-x7fvf" (UID: "9eb09939-3056-4ca9-8d22-64cd87cf4f8e") : secret "dns-default-metrics-tls" not found
Jun 03 16:34:43.746077 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:43.745888    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert podName:bd8eaa6c-3029-4458-9d0c-13933d35c3c6 nodeName:}" failed. No retries permitted until 2026-06-03 16:34:59.745872577 +0000 UTC m=+69.276392141 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert") pod "ingress-canary-tdqmj" (UID: "bd8eaa6c-3029-4458-9d0c-13933d35c3c6") : secret "canary-serving-cert" not found
Jun 03 16:34:52.325499 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:52.325473    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xh2c4"
Jun 03 16:34:56.738223 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:56.738163    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs\") pod \"network-metrics-daemon-q8x4r\" (UID: \"d1292e61-8cac-4672-af45-d434487b3426\") " pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:34:56.741246 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:56.741225    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jun 03 16:34:56.749014 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:56.748989    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jun 03 16:34:56.749076 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:56.749054    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs podName:d1292e61-8cac-4672-af45-d434487b3426 nodeName:}" failed. No retries permitted until 2026-06-03 16:36:00.749037769 +0000 UTC m=+130.279557330 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs") pod "network-metrics-daemon-q8x4r" (UID: "d1292e61-8cac-4672-af45-d434487b3426") : secret "metrics-daemon-secret" not found
Jun 03 16:34:56.839238 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:56.839197    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qfz5b\" (UniqueName: \"kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b\") pod \"network-check-target-bs44f\" (UID: \"cad8c088-209d-4676-b925-be649bcf3f51\") " pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:56.842284 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:56.842263    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jun 03 16:34:56.851721 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:56.851700    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jun 03 16:34:56.863074 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:56.863054    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfz5b\" (UniqueName: \"kubernetes.io/projected/cad8c088-209d-4676-b925-be649bcf3f51-kube-api-access-qfz5b\") pod \"network-check-target-bs44f\" (UID: \"cad8c088-209d-4676-b925-be649bcf3f51\") " pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:56.880458 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:56.880432    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r657c\""
Jun 03 16:34:56.888715 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:56.888694    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:34:57.005485 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:57.005416    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bs44f"]
Jun 03 16:34:57.008210 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:34:57.008184    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcad8c088_209d_4676_b925_be649bcf3f51.slice/crio-6954138525c3caac9a4fc6cc04f9f60dda184ec29ffab9bee03a472b4057b9b9 WatchSource:0}: Error finding container 6954138525c3caac9a4fc6cc04f9f60dda184ec29ffab9bee03a472b4057b9b9: Status 404 returned error can't find the container with id 6954138525c3caac9a4fc6cc04f9f60dda184ec29ffab9bee03a472b4057b9b9
Jun 03 16:34:57.436191 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:57.436155    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bs44f" event={"ID":"cad8c088-209d-4676-b925-be649bcf3f51","Type":"ContainerStarted","Data":"6954138525c3caac9a4fc6cc04f9f60dda184ec29ffab9bee03a472b4057b9b9"}
Jun 03 16:34:59.761699 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:59.761650    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:34:59.761699 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:34:59.761701    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert\") pod \"ingress-canary-tdqmj\" (UID: \"bd8eaa6c-3029-4458-9d0c-13933d35c3c6\") " pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:34:59.762115 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:59.761789    2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 03 16:34:59.762115 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:59.761822    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 16:34:59.762115 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:59.761871    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert podName:bd8eaa6c-3029-4458-9d0c-13933d35c3c6 nodeName:}" failed. No retries permitted until 2026-06-03 16:35:31.76185715 +0000 UTC m=+101.292376710 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert") pod "ingress-canary-tdqmj" (UID: "bd8eaa6c-3029-4458-9d0c-13933d35c3c6") : secret "canary-serving-cert" not found
Jun 03 16:34:59.762115 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:34:59.761885    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls podName:9eb09939-3056-4ca9-8d22-64cd87cf4f8e nodeName:}" failed. No retries permitted until 2026-06-03 16:35:31.761878932 +0000 UTC m=+101.292398493 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls") pod "dns-default-x7fvf" (UID: "9eb09939-3056-4ca9-8d22-64cd87cf4f8e") : secret "dns-default-metrics-tls" not found
Jun 03 16:35:00.446621 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:35:00.446589    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bs44f" event={"ID":"cad8c088-209d-4676-b925-be649bcf3f51","Type":"ContainerStarted","Data":"ae6d7f8013f7af6de3a4f1b40ad368d93e17320a2400318bdc83afaf6924bed3"}
Jun 03 16:35:00.446837 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:35:00.446808    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:35:00.462175 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:35:00.462130    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bs44f" podStartSLOduration=66.920418265 podStartE2EDuration="1m9.462119384s" podCreationTimestamp="2026-06-03 16:33:51 +0000 UTC" firstStartedPulling="2026-06-03 16:34:57.010138434 +0000 UTC m=+66.540657997" lastFinishedPulling="2026-06-03 16:34:59.551839551 +0000 UTC m=+69.082359116" observedRunningTime="2026-06-03 16:35:00.461081158 +0000 UTC m=+69.991600742" watchObservedRunningTime="2026-06-03 16:35:00.462119384 +0000 UTC m=+69.992638963"
Jun 03 16:35:31.450694 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:35:31.450665    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bs44f"
Jun 03 16:35:31.787230 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:35:31.787162    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:35:31.787230 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:35:31.787194    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert\") pod \"ingress-canary-tdqmj\" (UID: \"bd8eaa6c-3029-4458-9d0c-13933d35c3c6\") " pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:35:31.787414 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:35:31.787303    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 03 16:35:31.787414 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:35:31.787354    2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 03 16:35:31.787414 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:35:31.787370    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls podName:9eb09939-3056-4ca9-8d22-64cd87cf4f8e nodeName:}" failed. No retries permitted until 2026-06-03 16:36:35.787354541 +0000 UTC m=+165.317874102 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls") pod "dns-default-x7fvf" (UID: "9eb09939-3056-4ca9-8d22-64cd87cf4f8e") : secret "dns-default-metrics-tls" not found
Jun 03 16:35:31.787414 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:35:31.787392    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert podName:bd8eaa6c-3029-4458-9d0c-13933d35c3c6 nodeName:}" failed. No retries permitted until 2026-06-03 16:36:35.787380648 +0000 UTC m=+165.317900209 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert") pod "ingress-canary-tdqmj" (UID: "bd8eaa6c-3029-4458-9d0c-13933d35c3c6") : secret "canary-serving-cert" not found
Jun 03 16:35:56.236628 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:35:56.236598    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vzp5g_fefcdfd2-e498-45e1-ae6d-fa8431ba97fa/dns-node-resolver/0.log"
Jun 03 16:35:57.237152 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:35:57.237127    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ccbzr_c0fd9ccf-e728-4aba-9249-daed366df1e5/node-ca/0.log"
Jun 03 16:36:00.799067 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:00.799030    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs\") pod \"network-metrics-daemon-q8x4r\" (UID: \"d1292e61-8cac-4672-af45-d434487b3426\") " pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:36:00.799445 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:36:00.799170    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jun 03 16:36:00.799445 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:36:00.799246    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs podName:d1292e61-8cac-4672-af45-d434487b3426 nodeName:}" failed. No retries permitted until 2026-06-03 16:38:02.799228837 +0000 UTC m=+252.329748397 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs") pod "network-metrics-daemon-q8x4r" (UID: "d1292e61-8cac-4672-af45-d434487b3426") : secret "metrics-daemon-secret" not found
Jun 03 16:36:11.637697 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:11.637653    2569 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz" podUID="6b7e03c1-b306-4d27-8ea5-959a63034a79" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jun 03 16:36:17.340018 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.339982    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zx2cx"]
Jun 03 16:36:17.342976 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.342960    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.345782 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.345757    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jun 03 16:36:17.345782 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.345785    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jun 03 16:36:17.346944 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.346928    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jun 03 16:36:17.347245 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.347230    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jun 03 16:36:17.347288 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.347280    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m97v2\""
Jun 03 16:36:17.360610 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.360582    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zx2cx"]
Jun 03 16:36:17.414401 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.414372    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8czx\" (UniqueName: \"kubernetes.io/projected/636a1bf9-f7d4-4107-b669-30d08086d657-kube-api-access-q8czx\") pod \"insights-runtime-extractor-zx2cx\" (UID: \"636a1bf9-f7d4-4107-b669-30d08086d657\") " pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.414540 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.414415    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/636a1bf9-f7d4-4107-b669-30d08086d657-crio-socket\") pod \"insights-runtime-extractor-zx2cx\" (UID: \"636a1bf9-f7d4-4107-b669-30d08086d657\") " pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.414540 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.414438    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/636a1bf9-f7d4-4107-b669-30d08086d657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zx2cx\" (UID: \"636a1bf9-f7d4-4107-b669-30d08086d657\") " pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.414540 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.414500    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/636a1bf9-f7d4-4107-b669-30d08086d657-data-volume\") pod \"insights-runtime-extractor-zx2cx\" (UID: \"636a1bf9-f7d4-4107-b669-30d08086d657\") " pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.414540 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.414530    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/636a1bf9-f7d4-4107-b669-30d08086d657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zx2cx\" (UID: \"636a1bf9-f7d4-4107-b669-30d08086d657\") " pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.515436 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.515388    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q8czx\" (UniqueName: \"kubernetes.io/projected/636a1bf9-f7d4-4107-b669-30d08086d657-kube-api-access-q8czx\") pod \"insights-runtime-extractor-zx2cx\" (UID: \"636a1bf9-f7d4-4107-b669-30d08086d657\") " pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.515617 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.515446    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/636a1bf9-f7d4-4107-b669-30d08086d657-crio-socket\") pod \"insights-runtime-extractor-zx2cx\" (UID: \"636a1bf9-f7d4-4107-b669-30d08086d657\") " pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.515617 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.515467    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/636a1bf9-f7d4-4107-b669-30d08086d657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zx2cx\" (UID: \"636a1bf9-f7d4-4107-b669-30d08086d657\") " pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.515617 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.515494    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/636a1bf9-f7d4-4107-b669-30d08086d657-data-volume\") pod \"insights-runtime-extractor-zx2cx\" (UID: \"636a1bf9-f7d4-4107-b669-30d08086d657\") " pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.515617 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.515513    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/636a1bf9-f7d4-4107-b669-30d08086d657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zx2cx\" (UID: \"636a1bf9-f7d4-4107-b669-30d08086d657\") " pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.515617 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.515592    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/636a1bf9-f7d4-4107-b669-30d08086d657-crio-socket\") pod \"insights-runtime-extractor-zx2cx\" (UID: \"636a1bf9-f7d4-4107-b669-30d08086d657\") " pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.515892 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.515759    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/636a1bf9-f7d4-4107-b669-30d08086d657-data-volume\") pod \"insights-runtime-extractor-zx2cx\" (UID: \"636a1bf9-f7d4-4107-b669-30d08086d657\") " pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.516145 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.516110    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/636a1bf9-f7d4-4107-b669-30d08086d657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zx2cx\" (UID: \"636a1bf9-f7d4-4107-b669-30d08086d657\") " pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.517840 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.517825    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/636a1bf9-f7d4-4107-b669-30d08086d657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zx2cx\" (UID: \"636a1bf9-f7d4-4107-b669-30d08086d657\") " pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.526583 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.526566    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8czx\" (UniqueName: \"kubernetes.io/projected/636a1bf9-f7d4-4107-b669-30d08086d657-kube-api-access-q8czx\") pod \"insights-runtime-extractor-zx2cx\" (UID: \"636a1bf9-f7d4-4107-b669-30d08086d657\") " pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.651917 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.651783    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zx2cx"
Jun 03 16:36:17.769926 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:17.769833    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zx2cx"]
Jun 03 16:36:17.772753 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:36:17.772719    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod636a1bf9_f7d4_4107_b669_30d08086d657.slice/crio-1237d8b773d519a2f95167e54639611ec9be60e274a5747ab3e25e2bbd3e37ea WatchSource:0}: Error finding container 1237d8b773d519a2f95167e54639611ec9be60e274a5747ab3e25e2bbd3e37ea: Status 404 returned error can't find the container with id 1237d8b773d519a2f95167e54639611ec9be60e274a5747ab3e25e2bbd3e37ea
Jun 03 16:36:18.621075 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:18.621048    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zx2cx" event={"ID":"636a1bf9-f7d4-4107-b669-30d08086d657","Type":"ContainerStarted","Data":"a5c4af6bfcc6a4fbbdbc5ee29e2ae5b9e49de8f9ed871501ee10754161078223"}
Jun 03 16:36:18.621425 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:18.621082    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zx2cx" event={"ID":"636a1bf9-f7d4-4107-b669-30d08086d657","Type":"ContainerStarted","Data":"1237d8b773d519a2f95167e54639611ec9be60e274a5747ab3e25e2bbd3e37ea"}
Jun 03 16:36:19.625537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:19.625492    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zx2cx" event={"ID":"636a1bf9-f7d4-4107-b669-30d08086d657","Type":"ContainerStarted","Data":"e0fd7166020ab21421f7038d655a8f5ae573fa856dcdee6be51fb3a23df043b7"}
Jun 03 16:36:20.629624 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:20.629583    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zx2cx" event={"ID":"636a1bf9-f7d4-4107-b669-30d08086d657","Type":"ContainerStarted","Data":"f8fce225358ac94af285d61ff1c39726aee29159126d9e480e5feda99fa6869b"}
Jun 03 16:36:20.647729 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:20.647631    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zx2cx" podStartSLOduration=1.089323258 podStartE2EDuration="3.647616202s" podCreationTimestamp="2026-06-03 16:36:17 +0000 UTC" firstStartedPulling="2026-06-03 16:36:17.833055317 +0000 UTC m=+147.363574881" lastFinishedPulling="2026-06-03 16:36:20.39134826 +0000 UTC m=+149.921867825" observedRunningTime="2026-06-03 16:36:20.645698557 +0000 UTC m=+150.176218152" watchObservedRunningTime="2026-06-03 16:36:20.647616202 +0000 UTC m=+150.178135785"
Jun 03 16:36:21.637045 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:21.637012    2569 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz" podUID="6b7e03c1-b306-4d27-8ea5-959a63034a79" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jun 03 16:36:30.851779 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:36:30.851720    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-x7fvf" podUID="9eb09939-3056-4ca9-8d22-64cd87cf4f8e"
Jun 03 16:36:30.864919 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:36:30.864884    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-tdqmj" podUID="bd8eaa6c-3029-4458-9d0c-13933d35c3c6"
Jun 03 16:36:31.083945 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:36:31.083905    2569 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-q8x4r" podUID="d1292e61-8cac-4672-af45-d434487b3426"
Jun 03 16:36:31.637205 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.637159    2569 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz" podUID="6b7e03c1-b306-4d27-8ea5-959a63034a79" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jun 03 16:36:31.637365 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.637244    2569 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz"
Jun 03 16:36:31.637750 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.637718    2569 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"9389fc237ab53f668207a76f9554b6cc73baa09a9402786f1ece6947a3b94085"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jun 03 16:36:31.637846 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.637828    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz" podUID="6b7e03c1-b306-4d27-8ea5-959a63034a79" containerName="service-proxy" containerID="cri-o://9389fc237ab53f668207a76f9554b6cc73baa09a9402786f1ece6947a3b94085" gracePeriod=30
Jun 03 16:36:31.654000 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.653976    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:36:31.796771 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.796747    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-n5ql5"]
Jun 03 16:36:31.800028 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.800008    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:31.802564 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.802495    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jun 03 16:36:31.802683 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.802586    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jun 03 16:36:31.802683 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.802587    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jun 03 16:36:31.802914 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.802900    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jun 03 16:36:31.802981 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.802937    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jun 03 16:36:31.803924 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.803908    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jun 03 16:36:31.804018 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.803981    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-glg92\""
Jun 03 16:36:31.923506 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.923412    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-textfile\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:31.923506 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.923448    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44f9713b-30e4-4227-8250-ae656f6534bd-sys\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:31.923506 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.923478    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:31.923947 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.923510    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-wtmp\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:31.923947 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.923534    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szvsp\" (UniqueName: \"kubernetes.io/projected/44f9713b-30e4-4227-8250-ae656f6534bd-kube-api-access-szvsp\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:31.923947 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.923549    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44f9713b-30e4-4227-8250-ae656f6534bd-metrics-client-ca\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:31.923947 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.923684    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-tls\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:31.923947 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.923720    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/44f9713b-30e4-4227-8250-ae656f6534bd-root\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:31.923947 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:31.923747    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-accelerators-collector-config\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.024244 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.024207    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-wtmp\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.024244 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.024247    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-szvsp\" (UniqueName: \"kubernetes.io/projected/44f9713b-30e4-4227-8250-ae656f6534bd-kube-api-access-szvsp\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.024466 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.024271    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44f9713b-30e4-4227-8250-ae656f6534bd-metrics-client-ca\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.024466 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.024324    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-tls\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.024466 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.024353    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/44f9713b-30e4-4227-8250-ae656f6534bd-root\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.024466 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.024379    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-accelerators-collector-config\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.024466 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.024393    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-wtmp\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.024466 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.024408    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-textfile\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.024466 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.024434    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44f9713b-30e4-4227-8250-ae656f6534bd-sys\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.024466 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.024451    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/44f9713b-30e4-4227-8250-ae656f6534bd-root\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.024818 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:36:32.024470    2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jun 03 16:36:32.024818 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.024514    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44f9713b-30e4-4227-8250-ae656f6534bd-sys\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.024818 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.024480    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.024818 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:36:32.024547    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-tls podName:44f9713b-30e4-4227-8250-ae656f6534bd nodeName:}" failed. No retries permitted until 2026-06-03 16:36:32.524529281 +0000 UTC m=+162.055048845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-tls") pod "node-exporter-n5ql5" (UID: "44f9713b-30e4-4227-8250-ae656f6534bd") : secret "node-exporter-tls" not found
Jun 03 16:36:32.024977 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.024839    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-textfile\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.024977 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.024968    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44f9713b-30e4-4227-8250-ae656f6534bd-metrics-client-ca\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.025043 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.025029    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-accelerators-collector-config\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.026682 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.026665    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.033377 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.033353    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szvsp\" (UniqueName: \"kubernetes.io/projected/44f9713b-30e4-4227-8250-ae656f6534bd-kube-api-access-szvsp\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.528769 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.528732    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-tls\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.531003 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.530983    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/44f9713b-30e4-4227-8250-ae656f6534bd-node-exporter-tls\") pod \"node-exporter-n5ql5\" (UID: \"44f9713b-30e4-4227-8250-ae656f6534bd\") " pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.658737 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.658707    2569 generic.go:358] "Generic (PLEG): container finished" podID="6b7e03c1-b306-4d27-8ea5-959a63034a79" containerID="9389fc237ab53f668207a76f9554b6cc73baa09a9402786f1ece6947a3b94085" exitCode=2
Jun 03 16:36:32.658915 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.658749    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz" event={"ID":"6b7e03c1-b306-4d27-8ea5-959a63034a79","Type":"ContainerDied","Data":"9389fc237ab53f668207a76f9554b6cc73baa09a9402786f1ece6947a3b94085"}
Jun 03 16:36:32.658915 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.658771    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57cd466ff-tzvvz" event={"ID":"6b7e03c1-b306-4d27-8ea5-959a63034a79","Type":"ContainerStarted","Data":"1dda3f3c1f942d39c6a9065a8c3d3ff1a8e643584c4850cbc4bf537fac08abda"}
Jun 03 16:36:32.709866 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:32.709834    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n5ql5"
Jun 03 16:36:32.717474 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:36:32.717448    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44f9713b_30e4_4227_8250_ae656f6534bd.slice/crio-490a9e983aa677729832b7202263ece5018f850fa60020cd4d34134a0e9a0707 WatchSource:0}: Error finding container 490a9e983aa677729832b7202263ece5018f850fa60020cd4d34134a0e9a0707: Status 404 returned error can't find the container with id 490a9e983aa677729832b7202263ece5018f850fa60020cd4d34134a0e9a0707
Jun 03 16:36:33.662000 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:33.661965    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n5ql5" event={"ID":"44f9713b-30e4-4227-8250-ae656f6534bd","Type":"ContainerStarted","Data":"490a9e983aa677729832b7202263ece5018f850fa60020cd4d34134a0e9a0707"}
Jun 03 16:36:34.666725 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:34.666696    2569 generic.go:358] "Generic (PLEG): container finished" podID="44f9713b-30e4-4227-8250-ae656f6534bd" containerID="2d0be26e05e55dd28d2aa0c90be1ac640c0d73d382680f9b2e931f72fbf69eae" exitCode=0
Jun 03 16:36:34.667105 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:34.666775    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n5ql5" event={"ID":"44f9713b-30e4-4227-8250-ae656f6534bd","Type":"ContainerDied","Data":"2d0be26e05e55dd28d2aa0c90be1ac640c0d73d382680f9b2e931f72fbf69eae"}
Jun 03 16:36:35.671237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:35.671198    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n5ql5" event={"ID":"44f9713b-30e4-4227-8250-ae656f6534bd","Type":"ContainerStarted","Data":"3607495ec4e4f1ef8048d286c14f75385e63a1c9f49a8b7057b98ddbd2bbb789"}
Jun 03 16:36:35.671237 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:35.671236    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n5ql5" event={"ID":"44f9713b-30e4-4227-8250-ae656f6534bd","Type":"ContainerStarted","Data":"5b6c453a968425498c499b5bb1c1245ae5ab817069e1df473036ab914ee99cfb"}
Jun 03 16:36:35.692115 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:35.692068    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-n5ql5" podStartSLOduration=3.559466627 podStartE2EDuration="4.692055088s" podCreationTimestamp="2026-06-03 16:36:31 +0000 UTC" firstStartedPulling="2026-06-03 16:36:32.719170068 +0000 UTC m=+162.249689646" lastFinishedPulling="2026-06-03 16:36:33.851758543 +0000 UTC m=+163.382278107" observedRunningTime="2026-06-03 16:36:35.691287575 +0000 UTC m=+165.221807158" watchObservedRunningTime="2026-06-03 16:36:35.692055088 +0000 UTC m=+165.222574671"
Jun 03 16:36:35.855781 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:35.855739    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:36:35.855781 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:35.855780    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert\") pod \"ingress-canary-tdqmj\" (UID: \"bd8eaa6c-3029-4458-9d0c-13933d35c3c6\") " pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:36:35.858199 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:35.858172    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9eb09939-3056-4ca9-8d22-64cd87cf4f8e-metrics-tls\") pod \"dns-default-x7fvf\" (UID: \"9eb09939-3056-4ca9-8d22-64cd87cf4f8e\") " pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:36:35.858302 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:35.858259    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd8eaa6c-3029-4458-9d0c-13933d35c3c6-cert\") pod \"ingress-canary-tdqmj\" (UID: \"bd8eaa6c-3029-4458-9d0c-13933d35c3c6\") " pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:36:36.158091 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:36.158063    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6d9kw\""
Jun 03 16:36:36.165470 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:36.165449    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:36:36.281406 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:36.281380    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x7fvf"]
Jun 03 16:36:36.283344 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:36:36.283309    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb09939_3056_4ca9_8d22_64cd87cf4f8e.slice/crio-175ab974f6a59010904c54dbd584e61231972648b271760b1b226d968eb087f5 WatchSource:0}: Error finding container 175ab974f6a59010904c54dbd584e61231972648b271760b1b226d968eb087f5: Status 404 returned error can't find the container with id 175ab974f6a59010904c54dbd584e61231972648b271760b1b226d968eb087f5
Jun 03 16:36:36.674871 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:36.674839    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x7fvf" event={"ID":"9eb09939-3056-4ca9-8d22-64cd87cf4f8e","Type":"ContainerStarted","Data":"175ab974f6a59010904c54dbd584e61231972648b271760b1b226d968eb087f5"}
Jun 03 16:36:38.682655 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:38.682622    2569 generic.go:358] "Generic (PLEG): container finished" podID="071a20f8-777a-4d61-b6bd-b35e61799652" containerID="621547a172832f230736527834619fca5a9c5b000e88550004bbce8a8e8ba79e" exitCode=255
Jun 03 16:36:38.683123 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:38.682706    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76cf9dff44-d8hkh" event={"ID":"071a20f8-777a-4d61-b6bd-b35e61799652","Type":"ContainerDied","Data":"621547a172832f230736527834619fca5a9c5b000e88550004bbce8a8e8ba79e"}
Jun 03 16:36:38.683123 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:38.683054    2569 scope.go:117] "RemoveContainer" containerID="621547a172832f230736527834619fca5a9c5b000e88550004bbce8a8e8ba79e"
Jun 03 16:36:38.684331 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:38.684307    2569 generic.go:358] "Generic (PLEG): container finished" podID="6089ff9d-c8ed-4474-a288-c01b9f7baca9" containerID="602d5b2db8ff49656f28de4e125a4fa660f318eca7ac387f1ba876db3e758c46" exitCode=1
Jun 03 16:36:38.684440 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:38.684379    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr" event={"ID":"6089ff9d-c8ed-4474-a288-c01b9f7baca9","Type":"ContainerDied","Data":"602d5b2db8ff49656f28de4e125a4fa660f318eca7ac387f1ba876db3e758c46"}
Jun 03 16:36:38.684781 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:38.684715    2569 scope.go:117] "RemoveContainer" containerID="602d5b2db8ff49656f28de4e125a4fa660f318eca7ac387f1ba876db3e758c46"
Jun 03 16:36:38.686089 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:38.686048    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x7fvf" event={"ID":"9eb09939-3056-4ca9-8d22-64cd87cf4f8e","Type":"ContainerStarted","Data":"208fbcc6dc5a66531a7001ae9bda2f635adb625e1651d0e1355e063064ea9f1c"}
Jun 03 16:36:38.686089 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:38.686078    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x7fvf" event={"ID":"9eb09939-3056-4ca9-8d22-64cd87cf4f8e","Type":"ContainerStarted","Data":"010c00e26beb041a26da2731a7d8c28e461b5f31cfa9f731fb17a2c20af995a9"}
Jun 03 16:36:38.686235 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:38.686194    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:36:38.719012 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:38.718873    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-x7fvf" podStartSLOduration=129.902162871 podStartE2EDuration="2m11.718853589s" podCreationTimestamp="2026-06-03 16:34:27 +0000 UTC" firstStartedPulling="2026-06-03 16:36:36.285256447 +0000 UTC m=+165.815776023" lastFinishedPulling="2026-06-03 16:36:38.10194718 +0000 UTC m=+167.632466741" observedRunningTime="2026-06-03 16:36:38.718163592 +0000 UTC m=+168.248683175" watchObservedRunningTime="2026-06-03 16:36:38.718853589 +0000 UTC m=+168.249373175"
Jun 03 16:36:39.367730 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:39.367686    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"
Jun 03 16:36:39.689862 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:39.689765    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr" event={"ID":"6089ff9d-c8ed-4474-a288-c01b9f7baca9","Type":"ContainerStarted","Data":"2b00b120f1fe6e2d94bf1e4fb8babdf0ed153c9118fd83e182b8e22704e3a822"}
Jun 03 16:36:39.691351 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:39.691332    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76cf9dff44-d8hkh" event={"ID":"071a20f8-777a-4d61-b6bd-b35e61799652","Type":"ContainerStarted","Data":"867b3fdad65b3c2556943c88c6a3f01353a2b7fe703aaa142c1bd6c74c6103d4"}
Jun 03 16:36:40.694482 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:40.694447    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"
Jun 03 16:36:40.695142 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:40.695124    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cdf67d4b5-wqcvr"
Jun 03 16:36:41.661122 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.661089    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-f54695654-gn7wk"]
Jun 03 16:36:41.664154 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.664131    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.666951 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.666929    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jun 03 16:36:41.668333 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.668313    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6vt6b\""
Jun 03 16:36:41.668523 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.668316    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jun 03 16:36:41.668672 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.668541    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jun 03 16:36:41.668672 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.668335    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jun 03 16:36:41.668672 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.668573    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jun 03 16:36:41.668672 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.668547    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jun 03 16:36:41.668672 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.668394    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jun 03 16:36:41.674070 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.674051    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jun 03 16:36:41.674457 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.674439    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f54695654-gn7wk"]
Jun 03 16:36:41.803231 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.803188    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/491459f9-c728-4699-85d6-05872213453b-console-serving-cert\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.803231 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.803231    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq2x4\" (UniqueName: \"kubernetes.io/projected/491459f9-c728-4699-85d6-05872213453b-kube-api-access-jq2x4\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.803685 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.803258    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-trusted-ca-bundle\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.803685 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.803336    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-service-ca\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.803685 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.803390    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-console-config\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.803685 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.803421    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-oauth-serving-cert\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.803685 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.803479    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/491459f9-c728-4699-85d6-05872213453b-console-oauth-config\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.903828 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.903783    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/491459f9-c728-4699-85d6-05872213453b-console-oauth-config\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.903959 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.903867    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/491459f9-c728-4699-85d6-05872213453b-console-serving-cert\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.903959 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.903900    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jq2x4\" (UniqueName: \"kubernetes.io/projected/491459f9-c728-4699-85d6-05872213453b-kube-api-access-jq2x4\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.903959 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.903927    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-trusted-ca-bundle\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.903959 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.903953    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-service-ca\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.904114 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.903991    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-console-config\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.904425 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.904396    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-oauth-serving-cert\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.905417 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.905307    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-trusted-ca-bundle\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.905574 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.905521    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-service-ca\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.905574 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.905556    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-oauth-serving-cert\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.905691 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.905435    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-console-config\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.910146 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.910120    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/491459f9-c728-4699-85d6-05872213453b-console-serving-cert\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.911023 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.910997    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/491459f9-c728-4699-85d6-05872213453b-console-oauth-config\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.912416 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.912368    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq2x4\" (UniqueName: \"kubernetes.io/projected/491459f9-c728-4699-85d6-05872213453b-kube-api-access-jq2x4\") pod \"console-f54695654-gn7wk\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") " pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:41.974310 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:41.974268    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:42.093062 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:42.093035    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f54695654-gn7wk"]
Jun 03 16:36:42.095463 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:36:42.095436    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod491459f9_c728_4699_85d6_05872213453b.slice/crio-d266e9b97b629ddb8fa3b4c1c84aaa740cb8cfac5f11491fd0e8060a15f5c570 WatchSource:0}: Error finding container d266e9b97b629ddb8fa3b4c1c84aaa740cb8cfac5f11491fd0e8060a15f5c570: Status 404 returned error can't find the container with id d266e9b97b629ddb8fa3b4c1c84aaa740cb8cfac5f11491fd0e8060a15f5c570
Jun 03 16:36:42.702292 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:42.702250    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f54695654-gn7wk" event={"ID":"491459f9-c728-4699-85d6-05872213453b","Type":"ContainerStarted","Data":"d266e9b97b629ddb8fa3b4c1c84aaa740cb8cfac5f11491fd0e8060a15f5c570"}
Jun 03 16:36:44.068788 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:44.068758    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:36:45.068811 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:45.068770    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:36:45.071697 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:45.071678    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kldw8\""
Jun 03 16:36:45.079098 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:45.079071    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tdqmj"
Jun 03 16:36:45.204661 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:45.204578    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tdqmj"]
Jun 03 16:36:45.207199 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:36:45.207169    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd8eaa6c_3029_4458_9d0c_13933d35c3c6.slice/crio-c78e79a81a23e9e323fedb02ddbf9546459b444c36eae44c4f30856c7d9228a7 WatchSource:0}: Error finding container c78e79a81a23e9e323fedb02ddbf9546459b444c36eae44c4f30856c7d9228a7: Status 404 returned error can't find the container with id c78e79a81a23e9e323fedb02ddbf9546459b444c36eae44c4f30856c7d9228a7
Jun 03 16:36:45.713816 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:45.713761    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f54695654-gn7wk" event={"ID":"491459f9-c728-4699-85d6-05872213453b","Type":"ContainerStarted","Data":"e9fcc103da2a9848377d89c657227ca7f8c71093b7841b72960fe88873ea88ec"}
Jun 03 16:36:45.715141 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:45.715115    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tdqmj" event={"ID":"bd8eaa6c-3029-4458-9d0c-13933d35c3c6","Type":"ContainerStarted","Data":"c78e79a81a23e9e323fedb02ddbf9546459b444c36eae44c4f30856c7d9228a7"}
Jun 03 16:36:45.732115 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:45.732057    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-f54695654-gn7wk" podStartSLOduration=1.822983347 podStartE2EDuration="4.732040544s" podCreationTimestamp="2026-06-03 16:36:41 +0000 UTC" firstStartedPulling="2026-06-03 16:36:42.097450087 +0000 UTC m=+171.627969647" lastFinishedPulling="2026-06-03 16:36:45.00650728 +0000 UTC m=+174.537026844" observedRunningTime="2026-06-03 16:36:45.729753242 +0000 UTC m=+175.260272825" watchObservedRunningTime="2026-06-03 16:36:45.732040544 +0000 UTC m=+175.262560128"
Jun 03 16:36:47.721568 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:47.721530    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tdqmj" event={"ID":"bd8eaa6c-3029-4458-9d0c-13933d35c3c6","Type":"ContainerStarted","Data":"0a9ee955b1b4df9b8f5f7ff66c1b99f3f40a5c9abc574e1d0949638c345a44d1"}
Jun 03 16:36:47.737653 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:47.737598    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tdqmj" podStartSLOduration=139.258572143 podStartE2EDuration="2m20.737584815s" podCreationTimestamp="2026-06-03 16:34:27 +0000 UTC" firstStartedPulling="2026-06-03 16:36:45.209124018 +0000 UTC m=+174.739643582" lastFinishedPulling="2026-06-03 16:36:46.688136691 +0000 UTC m=+176.218656254" observedRunningTime="2026-06-03 16:36:47.736707172 +0000 UTC m=+177.267226746" watchObservedRunningTime="2026-06-03 16:36:47.737584815 +0000 UTC m=+177.268104397"
Jun 03 16:36:48.693737 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:48.693705    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x7fvf"
Jun 03 16:36:51.975052 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:51.975011    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:51.975052 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:51.975054    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:51.979612 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:51.979591    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:36:52.739131 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:36:52.739104    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:38:02.832033 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:02.831933    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs\") pod \"network-metrics-daemon-q8x4r\" (UID: \"d1292e61-8cac-4672-af45-d434487b3426\") " pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:38:02.834321 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:02.834297    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1292e61-8cac-4672-af45-d434487b3426-metrics-certs\") pod \"network-metrics-daemon-q8x4r\" (UID: \"d1292e61-8cac-4672-af45-d434487b3426\") " pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:38:02.972905 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:02.972872    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fx9bm\""
Jun 03 16:38:02.980071 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:02.980047    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q8x4r"
Jun 03 16:38:03.118460 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:03.118375    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-q8x4r"]
Jun 03 16:38:03.121332 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:38:03.121295    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1292e61_8cac_4672_af45_d434487b3426.slice/crio-c870447be0b9fa39e36adeeab00d67ca94d411833dd33a6f1e9782b5842f390a WatchSource:0}: Error finding container c870447be0b9fa39e36adeeab00d67ca94d411833dd33a6f1e9782b5842f390a: Status 404 returned error can't find the container with id c870447be0b9fa39e36adeeab00d67ca94d411833dd33a6f1e9782b5842f390a
Jun 03 16:38:03.913336 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:03.913290    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q8x4r" event={"ID":"d1292e61-8cac-4672-af45-d434487b3426","Type":"ContainerStarted","Data":"c870447be0b9fa39e36adeeab00d67ca94d411833dd33a6f1e9782b5842f390a"}
Jun 03 16:38:04.917279 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:04.917243    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q8x4r" event={"ID":"d1292e61-8cac-4672-af45-d434487b3426","Type":"ContainerStarted","Data":"b5f09cdd491df7eade827f0507ea34a1ae7543e1e8f63c92c333f2155108e3b1"}
Jun 03 16:38:04.917279 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:04.917281    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q8x4r" event={"ID":"d1292e61-8cac-4672-af45-d434487b3426","Type":"ContainerStarted","Data":"ec58e01bb2dcba7613686edaf1c618326b467ffbea3090bfb213e0da48bfe411"}
Jun 03 16:38:04.933334 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:04.933286    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-q8x4r" podStartSLOduration=253.064021772 podStartE2EDuration="4m13.933271779s" podCreationTimestamp="2026-06-03 16:33:51 +0000 UTC" firstStartedPulling="2026-06-03 16:38:03.123167356 +0000 UTC m=+252.653686917" lastFinishedPulling="2026-06-03 16:38:03.992417363 +0000 UTC m=+253.522936924" observedRunningTime="2026-06-03 16:38:04.932414707 +0000 UTC m=+254.462934308" watchObservedRunningTime="2026-06-03 16:38:04.933271779 +0000 UTC m=+254.463791363"
Jun 03 16:38:12.337478 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:12.337441    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f54695654-gn7wk"]
Jun 03 16:38:37.958862 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:37.958784    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-f54695654-gn7wk" podUID="491459f9-c728-4699-85d6-05872213453b" containerName="console" containerID="cri-o://e9fcc103da2a9848377d89c657227ca7f8c71093b7841b72960fe88873ea88ec" gracePeriod=15
Jun 03 16:38:38.194643 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.194623    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-f54695654-gn7wk_491459f9-c728-4699-85d6-05872213453b/console/0.log"
Jun 03 16:38:38.194757 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.194699    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:38:38.279761 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.279677    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-console-config\") pod \"491459f9-c728-4699-85d6-05872213453b\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") "
Jun 03 16:38:38.279761 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.279728    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-trusted-ca-bundle\") pod \"491459f9-c728-4699-85d6-05872213453b\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") "
Jun 03 16:38:38.279761 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.279755    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/491459f9-c728-4699-85d6-05872213453b-console-oauth-config\") pod \"491459f9-c728-4699-85d6-05872213453b\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") "
Jun 03 16:38:38.280056 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.279962    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/491459f9-c728-4699-85d6-05872213453b-console-serving-cert\") pod \"491459f9-c728-4699-85d6-05872213453b\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") "
Jun 03 16:38:38.280056 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.280029    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jq2x4\" (UniqueName: \"kubernetes.io/projected/491459f9-c728-4699-85d6-05872213453b-kube-api-access-jq2x4\") pod \"491459f9-c728-4699-85d6-05872213453b\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") "
Jun 03 16:38:38.280162 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.280057    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-service-ca\") pod \"491459f9-c728-4699-85d6-05872213453b\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") "
Jun 03 16:38:38.280162 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.280106    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-oauth-serving-cert\") pod \"491459f9-c728-4699-85d6-05872213453b\" (UID: \"491459f9-c728-4699-85d6-05872213453b\") "
Jun 03 16:38:38.280253 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.280178    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-console-config" (OuterVolumeSpecName: "console-config") pod "491459f9-c728-4699-85d6-05872213453b" (UID: "491459f9-c728-4699-85d6-05872213453b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 16:38:38.280253 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.280190    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "491459f9-c728-4699-85d6-05872213453b" (UID: "491459f9-c728-4699-85d6-05872213453b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 16:38:38.280450 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.280428    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-console-config\") on node \"ip-10-0-140-123.ec2.internal\" DevicePath \"\""
Jun 03 16:38:38.280534 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.280455    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-trusted-ca-bundle\") on node \"ip-10-0-140-123.ec2.internal\" DevicePath \"\""
Jun 03 16:38:38.280534 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.280485    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-service-ca" (OuterVolumeSpecName: "service-ca") pod "491459f9-c728-4699-85d6-05872213453b" (UID: "491459f9-c728-4699-85d6-05872213453b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 16:38:38.280618 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.280541    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "491459f9-c728-4699-85d6-05872213453b" (UID: "491459f9-c728-4699-85d6-05872213453b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 03 16:38:38.282034 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.282010    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/491459f9-c728-4699-85d6-05872213453b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "491459f9-c728-4699-85d6-05872213453b" (UID: "491459f9-c728-4699-85d6-05872213453b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 03 16:38:38.282131 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.282098    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/491459f9-c728-4699-85d6-05872213453b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "491459f9-c728-4699-85d6-05872213453b" (UID: "491459f9-c728-4699-85d6-05872213453b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 03 16:38:38.282191 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.282172    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/491459f9-c728-4699-85d6-05872213453b-kube-api-access-jq2x4" (OuterVolumeSpecName: "kube-api-access-jq2x4") pod "491459f9-c728-4699-85d6-05872213453b" (UID: "491459f9-c728-4699-85d6-05872213453b"). InnerVolumeSpecName "kube-api-access-jq2x4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 16:38:38.381060 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.381024    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/491459f9-c728-4699-85d6-05872213453b-console-oauth-config\") on node \"ip-10-0-140-123.ec2.internal\" DevicePath \"\""
Jun 03 16:38:38.381060 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.381053    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/491459f9-c728-4699-85d6-05872213453b-console-serving-cert\") on node \"ip-10-0-140-123.ec2.internal\" DevicePath \"\""
Jun 03 16:38:38.381060 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.381063    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jq2x4\" (UniqueName: \"kubernetes.io/projected/491459f9-c728-4699-85d6-05872213453b-kube-api-access-jq2x4\") on node \"ip-10-0-140-123.ec2.internal\" DevicePath \"\""
Jun 03 16:38:38.381060 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.381072    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-service-ca\") on node \"ip-10-0-140-123.ec2.internal\" DevicePath \"\""
Jun 03 16:38:38.381325 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:38.381081    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/491459f9-c728-4699-85d6-05872213453b-oauth-serving-cert\") on node \"ip-10-0-140-123.ec2.internal\" DevicePath \"\""
Jun 03 16:38:39.008462 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:39.008436    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-f54695654-gn7wk_491459f9-c728-4699-85d6-05872213453b/console/0.log"
Jun 03 16:38:39.008887 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:39.008478    2569 generic.go:358] "Generic (PLEG): container finished" podID="491459f9-c728-4699-85d6-05872213453b" containerID="e9fcc103da2a9848377d89c657227ca7f8c71093b7841b72960fe88873ea88ec" exitCode=2
Jun 03 16:38:39.008887 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:39.008517    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f54695654-gn7wk" event={"ID":"491459f9-c728-4699-85d6-05872213453b","Type":"ContainerDied","Data":"e9fcc103da2a9848377d89c657227ca7f8c71093b7841b72960fe88873ea88ec"}
Jun 03 16:38:39.008887 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:39.008553    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f54695654-gn7wk"
Jun 03 16:38:39.008887 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:39.008559    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f54695654-gn7wk" event={"ID":"491459f9-c728-4699-85d6-05872213453b","Type":"ContainerDied","Data":"d266e9b97b629ddb8fa3b4c1c84aaa740cb8cfac5f11491fd0e8060a15f5c570"}
Jun 03 16:38:39.008887 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:39.008575    2569 scope.go:117] "RemoveContainer" containerID="e9fcc103da2a9848377d89c657227ca7f8c71093b7841b72960fe88873ea88ec"
Jun 03 16:38:39.016351 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:39.016332    2569 scope.go:117] "RemoveContainer" containerID="e9fcc103da2a9848377d89c657227ca7f8c71093b7841b72960fe88873ea88ec"
Jun 03 16:38:39.016607 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:38:39.016588    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9fcc103da2a9848377d89c657227ca7f8c71093b7841b72960fe88873ea88ec\": container with ID starting with e9fcc103da2a9848377d89c657227ca7f8c71093b7841b72960fe88873ea88ec not found: ID does not exist" containerID="e9fcc103da2a9848377d89c657227ca7f8c71093b7841b72960fe88873ea88ec"
Jun 03 16:38:39.016650 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:39.016616    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9fcc103da2a9848377d89c657227ca7f8c71093b7841b72960fe88873ea88ec"} err="failed to get container status \"e9fcc103da2a9848377d89c657227ca7f8c71093b7841b72960fe88873ea88ec\": rpc error: code = NotFound desc = could not find container \"e9fcc103da2a9848377d89c657227ca7f8c71093b7841b72960fe88873ea88ec\": container with ID starting with e9fcc103da2a9848377d89c657227ca7f8c71093b7841b72960fe88873ea88ec not found: ID does not exist"
Jun 03 16:38:39.028910 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:39.028886    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f54695654-gn7wk"]
Jun 03 16:38:39.032097 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:39.032078    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f54695654-gn7wk"]
Jun 03 16:38:39.072223 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:39.072195    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="491459f9-c728-4699-85d6-05872213453b" path="/var/lib/kubelet/pods/491459f9-c728-4699-85d6-05872213453b/volumes"
Jun 03 16:38:50.954542 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:38:50.954515    2569 kubelet.go:1628] "Image garbage collection succeeded"
Jun 03 16:40:08.943957 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:08.943918    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-gdcqw"]
Jun 03 16:40:08.944600 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:08.944234    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="491459f9-c728-4699-85d6-05872213453b" containerName="console"
Jun 03 16:40:08.944600 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:08.944250    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="491459f9-c728-4699-85d6-05872213453b" containerName="console"
Jun 03 16:40:08.944600 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:08.944307    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="491459f9-c728-4699-85d6-05872213453b" containerName="console"
Jun 03 16:40:08.946988 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:08.946970    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gdcqw"
Jun 03 16:40:08.949944 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:08.949921    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jun 03 16:40:08.950024 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:08.949951    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jun 03 16:40:08.951185 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:08.951170    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-jhv7p\""
Jun 03 16:40:08.953979 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:08.953961    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-gdcqw"]
Jun 03 16:40:09.005280 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:09.005257    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s26n2\" (UniqueName: \"kubernetes.io/projected/31b191d4-a9b4-40a8-bca3-a9b538f648a8-kube-api-access-s26n2\") pod \"openshift-lws-operator-bfc7f696d-gdcqw\" (UID: \"31b191d4-a9b4-40a8-bca3-a9b538f648a8\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gdcqw"
Jun 03 16:40:09.005410 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:09.005299    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31b191d4-a9b4-40a8-bca3-a9b538f648a8-tmp\") pod \"openshift-lws-operator-bfc7f696d-gdcqw\" (UID: \"31b191d4-a9b4-40a8-bca3-a9b538f648a8\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gdcqw"
Jun 03 16:40:09.106089 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:09.106061    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s26n2\" (UniqueName: \"kubernetes.io/projected/31b191d4-a9b4-40a8-bca3-a9b538f648a8-kube-api-access-s26n2\") pod \"openshift-lws-operator-bfc7f696d-gdcqw\" (UID: \"31b191d4-a9b4-40a8-bca3-a9b538f648a8\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gdcqw"
Jun 03 16:40:09.106232 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:09.106111    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31b191d4-a9b4-40a8-bca3-a9b538f648a8-tmp\") pod \"openshift-lws-operator-bfc7f696d-gdcqw\" (UID: \"31b191d4-a9b4-40a8-bca3-a9b538f648a8\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gdcqw"
Jun 03 16:40:09.106430 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:09.106411    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31b191d4-a9b4-40a8-bca3-a9b538f648a8-tmp\") pod \"openshift-lws-operator-bfc7f696d-gdcqw\" (UID: \"31b191d4-a9b4-40a8-bca3-a9b538f648a8\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gdcqw"
Jun 03 16:40:09.114462 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:09.114436    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s26n2\" (UniqueName: \"kubernetes.io/projected/31b191d4-a9b4-40a8-bca3-a9b538f648a8-kube-api-access-s26n2\") pod \"openshift-lws-operator-bfc7f696d-gdcqw\" (UID: \"31b191d4-a9b4-40a8-bca3-a9b538f648a8\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gdcqw"
Jun 03 16:40:09.256682 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:09.256608    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gdcqw"
Jun 03 16:40:09.377828 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:09.377750    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-gdcqw"]
Jun 03 16:40:09.380051 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:40:09.380025    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31b191d4_a9b4_40a8_bca3_a9b538f648a8.slice/crio-ade5360c54408632657fab0343e70841c6f755f02044d38ef95130e3e30fbad9 WatchSource:0}: Error finding container ade5360c54408632657fab0343e70841c6f755f02044d38ef95130e3e30fbad9: Status 404 returned error can't find the container with id ade5360c54408632657fab0343e70841c6f755f02044d38ef95130e3e30fbad9
Jun 03 16:40:09.382374 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:09.382355    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 03 16:40:10.237093 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:10.237050    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gdcqw" event={"ID":"31b191d4-a9b4-40a8-bca3-a9b538f648a8","Type":"ContainerStarted","Data":"ade5360c54408632657fab0343e70841c6f755f02044d38ef95130e3e30fbad9"}
Jun 03 16:40:13.245754 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:13.245678    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gdcqw" event={"ID":"31b191d4-a9b4-40a8-bca3-a9b538f648a8","Type":"ContainerStarted","Data":"d44a69d60a95314bba7dc0cf480c35cd13e9a10d2d2ace23f19ea81e252542c4"}
Jun 03 16:40:13.264437 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:13.264383    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gdcqw" podStartSLOduration=1.775716887 podStartE2EDuration="5.264365931s" podCreationTimestamp="2026-06-03 16:40:08 +0000 UTC" firstStartedPulling="2026-06-03 16:40:09.382516453 +0000 UTC m=+378.913036014" lastFinishedPulling="2026-06-03 16:40:12.871165494 +0000 UTC m=+382.401685058" observedRunningTime="2026-06-03 16:40:13.262711303 +0000 UTC m=+382.793230885" watchObservedRunningTime="2026-06-03 16:40:13.264365931 +0000 UTC m=+382.794885516"
Jun 03 16:40:32.238379 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.238344    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"]
Jun 03 16:40:32.244530 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.244512    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.250717 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.250490    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jun 03 16:40:32.250717 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.250490    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jun 03 16:40:32.250717 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.250625    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zrctl\""
Jun 03 16:40:32.250921 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.250758    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jun 03 16:40:32.251716 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.251696    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jun 03 16:40:32.262682 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.262657    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"]
Jun 03 16:40:32.263963 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.263944    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhhxh\" (UniqueName: \"kubernetes.io/projected/4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a-kube-api-access-nhhxh\") pod \"opendatahub-operator-controller-manager-5f84758474-skksw\" (UID: \"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.264040 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.263976    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a-manifests\") pod \"opendatahub-operator-controller-manager-5f84758474-skksw\" (UID: \"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.264040 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.263996    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a-tmp\") pod \"opendatahub-operator-controller-manager-5f84758474-skksw\" (UID: \"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.264040 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.264014    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f84758474-skksw\" (UID: \"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.264172 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.264121    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f84758474-skksw\" (UID: \"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.364669 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.364645    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f84758474-skksw\" (UID: \"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.364857 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.364679    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nhhxh\" (UniqueName: \"kubernetes.io/projected/4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a-kube-api-access-nhhxh\") pod \"opendatahub-operator-controller-manager-5f84758474-skksw\" (UID: \"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.364857 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.364701    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a-manifests\") pod \"opendatahub-operator-controller-manager-5f84758474-skksw\" (UID: \"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.364857 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.364719    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a-tmp\") pod \"opendatahub-operator-controller-manager-5f84758474-skksw\" (UID: \"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.364857 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.364748    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f84758474-skksw\" (UID: \"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.365219 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.365194    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a-manifests\") pod \"opendatahub-operator-controller-manager-5f84758474-skksw\" (UID: \"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.366965 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.366938    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a-tmp\") pod \"opendatahub-operator-controller-manager-5f84758474-skksw\" (UID: \"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.367125 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.367103    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f84758474-skksw\" (UID: \"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.367189 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.367146    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f84758474-skksw\" (UID: \"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.375286 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.375266    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhhxh\" (UniqueName: \"kubernetes.io/projected/4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a-kube-api-access-nhhxh\") pod \"opendatahub-operator-controller-manager-5f84758474-skksw\" (UID: \"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.554310 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.554279    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:32.690843 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:32.690814    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"]
Jun 03 16:40:32.693650 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:40:32.693617    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a4bbbec_0955_4a88_b1ed_ed5a6b1f011a.slice/crio-8b62fec147fd5aabd5a54b7a19abb9cbe77bc641aed0de330ef2d538b121823b WatchSource:0}: Error finding container 8b62fec147fd5aabd5a54b7a19abb9cbe77bc641aed0de330ef2d538b121823b: Status 404 returned error can't find the container with id 8b62fec147fd5aabd5a54b7a19abb9cbe77bc641aed0de330ef2d538b121823b
Jun 03 16:40:33.301027 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:33.300978    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw" event={"ID":"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a","Type":"ContainerStarted","Data":"8b62fec147fd5aabd5a54b7a19abb9cbe77bc641aed0de330ef2d538b121823b"}
Jun 03 16:40:36.312476 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:36.312445    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw" event={"ID":"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a","Type":"ContainerStarted","Data":"4ba034e2e687d1521c2b65d2c68b6b5aab29a5f2c98d22e9f77031a03fc50193"}
Jun 03 16:40:37.316001 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:37.315964    2569 generic.go:358] "Generic (PLEG): container finished" podID="4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a" containerID="4ba034e2e687d1521c2b65d2c68b6b5aab29a5f2c98d22e9f77031a03fc50193" exitCode=0
Jun 03 16:40:37.316394 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:37.316067    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw" event={"ID":"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a","Type":"ContainerDied","Data":"4ba034e2e687d1521c2b65d2c68b6b5aab29a5f2c98d22e9f77031a03fc50193"}
Jun 03 16:40:38.320723 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:38.320691    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw" event={"ID":"4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a","Type":"ContainerStarted","Data":"02d11b4c53e44fc40cf5f9c85e0d72826989d9ec81bebdfd89b4676a37b2f98c"}
Jun 03 16:40:38.321111 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:38.320825    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:38.357429 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:38.357369    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw" podStartSLOduration=1.38099291 podStartE2EDuration="6.35735086s" podCreationTimestamp="2026-06-03 16:40:32 +0000 UTC" firstStartedPulling="2026-06-03 16:40:32.695693055 +0000 UTC m=+402.226212619" lastFinishedPulling="2026-06-03 16:40:37.672050993 +0000 UTC m=+407.202570569" observedRunningTime="2026-06-03 16:40:38.353638722 +0000 UTC m=+407.884158305" watchObservedRunningTime="2026-06-03 16:40:38.35735086 +0000 UTC m=+407.887870444"
Jun 03 16:40:49.325433 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:49.325404    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5f84758474-skksw"
Jun 03 16:40:51.878863 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:51.878832    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-76588b95-5f8mp"]
Jun 03 16:40:51.887096 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:51.887071    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-5f8mp"
Jun 03 16:40:51.890017 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:51.889993    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jun 03 16:40:51.890873 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:51.890848    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-sb2qv\""
Jun 03 16:40:51.890955 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:51.890856    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-5f8mp"]
Jun 03 16:40:52.004211 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.004178    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbqhf\" (UniqueName: \"kubernetes.io/projected/c2f6ac50-09f1-4f61-8160-530a2102718b-kube-api-access-sbqhf\") pod \"odh-model-controller-76588b95-5f8mp\" (UID: \"c2f6ac50-09f1-4f61-8160-530a2102718b\") " pod="opendatahub/odh-model-controller-76588b95-5f8mp"
Jun 03 16:40:52.004387 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.004233    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2f6ac50-09f1-4f61-8160-530a2102718b-cert\") pod \"odh-model-controller-76588b95-5f8mp\" (UID: \"c2f6ac50-09f1-4f61-8160-530a2102718b\") " pod="opendatahub/odh-model-controller-76588b95-5f8mp"
Jun 03 16:40:52.105577 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.105534    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2f6ac50-09f1-4f61-8160-530a2102718b-cert\") pod \"odh-model-controller-76588b95-5f8mp\" (UID: \"c2f6ac50-09f1-4f61-8160-530a2102718b\") " pod="opendatahub/odh-model-controller-76588b95-5f8mp"
Jun 03 16:40:52.105749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.105601    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sbqhf\" (UniqueName: \"kubernetes.io/projected/c2f6ac50-09f1-4f61-8160-530a2102718b-kube-api-access-sbqhf\") pod \"odh-model-controller-76588b95-5f8mp\" (UID: \"c2f6ac50-09f1-4f61-8160-530a2102718b\") " pod="opendatahub/odh-model-controller-76588b95-5f8mp"
Jun 03 16:40:52.105749 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:40:52.105697    2569 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jun 03 16:40:52.105873 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:40:52.105781    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2f6ac50-09f1-4f61-8160-530a2102718b-cert podName:c2f6ac50-09f1-4f61-8160-530a2102718b nodeName:}" failed. No retries permitted until 2026-06-03 16:40:52.605761069 +0000 UTC m=+422.136280631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c2f6ac50-09f1-4f61-8160-530a2102718b-cert") pod "odh-model-controller-76588b95-5f8mp" (UID: "c2f6ac50-09f1-4f61-8160-530a2102718b") : secret "odh-model-controller-webhook-cert" not found
Jun 03 16:40:52.130657 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.130594    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbqhf\" (UniqueName: \"kubernetes.io/projected/c2f6ac50-09f1-4f61-8160-530a2102718b-kube-api-access-sbqhf\") pod \"odh-model-controller-76588b95-5f8mp\" (UID: \"c2f6ac50-09f1-4f61-8160-530a2102718b\") " pod="opendatahub/odh-model-controller-76588b95-5f8mp"
Jun 03 16:40:52.554831 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.554784    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-6577596cd5-kqzpv"]
Jun 03 16:40:52.558206 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.558186    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-6577596cd5-kqzpv"
Jun 03 16:40:52.561156 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.561134    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-5t7d2\""
Jun 03 16:40:52.561156 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.561149    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jun 03 16:40:52.570097 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.570069    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-6577596cd5-kqzpv"]
Jun 03 16:40:52.608941 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.608914    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2f6ac50-09f1-4f61-8160-530a2102718b-cert\") pod \"odh-model-controller-76588b95-5f8mp\" (UID: \"c2f6ac50-09f1-4f61-8160-530a2102718b\") " pod="opendatahub/odh-model-controller-76588b95-5f8mp"
Jun 03 16:40:52.609088 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.608951    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e25b10b0-b604-43a8-9d61-9ef957201a7e-tls-certs\") pod \"model-serving-api-6577596cd5-kqzpv\" (UID: \"e25b10b0-b604-43a8-9d61-9ef957201a7e\") " pod="opendatahub/model-serving-api-6577596cd5-kqzpv"
Jun 03 16:40:52.609088 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.608996    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwb2m\" (UniqueName: \"kubernetes.io/projected/e25b10b0-b604-43a8-9d61-9ef957201a7e-kube-api-access-lwb2m\") pod \"model-serving-api-6577596cd5-kqzpv\" (UID: \"e25b10b0-b604-43a8-9d61-9ef957201a7e\") " pod="opendatahub/model-serving-api-6577596cd5-kqzpv"
Jun 03 16:40:52.611359 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.611328    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2f6ac50-09f1-4f61-8160-530a2102718b-cert\") pod \"odh-model-controller-76588b95-5f8mp\" (UID: \"c2f6ac50-09f1-4f61-8160-530a2102718b\") " pod="opendatahub/odh-model-controller-76588b95-5f8mp"
Jun 03 16:40:52.709477 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.709442    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e25b10b0-b604-43a8-9d61-9ef957201a7e-tls-certs\") pod \"model-serving-api-6577596cd5-kqzpv\" (UID: \"e25b10b0-b604-43a8-9d61-9ef957201a7e\") " pod="opendatahub/model-serving-api-6577596cd5-kqzpv"
Jun 03 16:40:52.709633 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.709493    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lwb2m\" (UniqueName: \"kubernetes.io/projected/e25b10b0-b604-43a8-9d61-9ef957201a7e-kube-api-access-lwb2m\") pod \"model-serving-api-6577596cd5-kqzpv\" (UID: \"e25b10b0-b604-43a8-9d61-9ef957201a7e\") " pod="opendatahub/model-serving-api-6577596cd5-kqzpv"
Jun 03 16:40:52.711876 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.711856    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e25b10b0-b604-43a8-9d61-9ef957201a7e-tls-certs\") pod \"model-serving-api-6577596cd5-kqzpv\" (UID: \"e25b10b0-b604-43a8-9d61-9ef957201a7e\") " pod="opendatahub/model-serving-api-6577596cd5-kqzpv"
Jun 03 16:40:52.718432 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.718407    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwb2m\" (UniqueName: \"kubernetes.io/projected/e25b10b0-b604-43a8-9d61-9ef957201a7e-kube-api-access-lwb2m\") pod \"model-serving-api-6577596cd5-kqzpv\" (UID: \"e25b10b0-b604-43a8-9d61-9ef957201a7e\") " pod="opendatahub/model-serving-api-6577596cd5-kqzpv"
Jun 03 16:40:52.798579 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.798549    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-5f8mp"
Jun 03 16:40:52.868280 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.868247    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-6577596cd5-kqzpv"
Jun 03 16:40:52.926160 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.926127    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-5f8mp"]
Jun 03 16:40:52.930471 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:40:52.930443    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2f6ac50_09f1_4f61_8160_530a2102718b.slice/crio-05163859d9f0fada7b7588688251939702615913265544aa1e5865a279e472c3 WatchSource:0}: Error finding container 05163859d9f0fada7b7588688251939702615913265544aa1e5865a279e472c3: Status 404 returned error can't find the container with id 05163859d9f0fada7b7588688251939702615913265544aa1e5865a279e472c3
Jun 03 16:40:52.949107 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.949080    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn"]
Jun 03 16:40:52.954236 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.954177    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn"
Jun 03 16:40:52.957170 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.957150    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xs9ww\""
Jun 03 16:40:52.957503 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.957486    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jun 03 16:40:52.957688 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.957673    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jun 03 16:40:52.957937 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.957920    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jun 03 16:40:52.958566 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.958507    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jun 03 16:40:52.967916 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:52.962302    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn"]
Jun 03 16:40:53.001762 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:53.001741    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-6577596cd5-kqzpv"]
Jun 03 16:40:53.004013 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:40:53.003990    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode25b10b0_b604_43a8_9d61_9ef957201a7e.slice/crio-f0fa2d19e5f0347e833af6fd61d8f7f0c0c9bbcfdbf5f6b9b23ce10fd6163709 WatchSource:0}: Error finding container f0fa2d19e5f0347e833af6fd61d8f7f0c0c9bbcfdbf5f6b9b23ce10fd6163709: Status 404 returned error can't find the container with id f0fa2d19e5f0347e833af6fd61d8f7f0c0c9bbcfdbf5f6b9b23ce10fd6163709
Jun 03 16:40:53.012383 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:53.012359    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gf9x\" (UniqueName: \"kubernetes.io/projected/6710ee7b-cbd5-4791-95c0-2c31b54e66b2-kube-api-access-8gf9x\") pod \"kube-auth-proxy-598f4f7759-pv7cn\" (UID: \"6710ee7b-cbd5-4791-95c0-2c31b54e66b2\") " pod="openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn"
Jun 03 16:40:53.012490 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:53.012395    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6710ee7b-cbd5-4791-95c0-2c31b54e66b2-tls-certs\") pod \"kube-auth-proxy-598f4f7759-pv7cn\" (UID: \"6710ee7b-cbd5-4791-95c0-2c31b54e66b2\") " pod="openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn"
Jun 03 16:40:53.012546 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:53.012487    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6710ee7b-cbd5-4791-95c0-2c31b54e66b2-tmp\") pod \"kube-auth-proxy-598f4f7759-pv7cn\" (UID: \"6710ee7b-cbd5-4791-95c0-2c31b54e66b2\") " pod="openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn"
Jun 03 16:40:53.113360 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:53.113281    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8gf9x\" (UniqueName: \"kubernetes.io/projected/6710ee7b-cbd5-4791-95c0-2c31b54e66b2-kube-api-access-8gf9x\") pod \"kube-auth-proxy-598f4f7759-pv7cn\" (UID: \"6710ee7b-cbd5-4791-95c0-2c31b54e66b2\") " pod="openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn"
Jun 03 16:40:53.113360 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:53.113314    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6710ee7b-cbd5-4791-95c0-2c31b54e66b2-tls-certs\") pod \"kube-auth-proxy-598f4f7759-pv7cn\" (UID: \"6710ee7b-cbd5-4791-95c0-2c31b54e66b2\") " pod="openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn"
Jun 03 16:40:53.113586 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:53.113451    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6710ee7b-cbd5-4791-95c0-2c31b54e66b2-tmp\") pod \"kube-auth-proxy-598f4f7759-pv7cn\" (UID: \"6710ee7b-cbd5-4791-95c0-2c31b54e66b2\") " pod="openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn"
Jun 03 16:40:53.115639 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:53.115603    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6710ee7b-cbd5-4791-95c0-2c31b54e66b2-tmp\") pod \"kube-auth-proxy-598f4f7759-pv7cn\" (UID: \"6710ee7b-cbd5-4791-95c0-2c31b54e66b2\") " pod="openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn"
Jun 03 16:40:53.115777 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:53.115762    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6710ee7b-cbd5-4791-95c0-2c31b54e66b2-tls-certs\") pod \"kube-auth-proxy-598f4f7759-pv7cn\" (UID: \"6710ee7b-cbd5-4791-95c0-2c31b54e66b2\") " pod="openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn"
Jun 03 16:40:53.124502 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:53.124483    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gf9x\" (UniqueName: \"kubernetes.io/projected/6710ee7b-cbd5-4791-95c0-2c31b54e66b2-kube-api-access-8gf9x\") pod \"kube-auth-proxy-598f4f7759-pv7cn\" (UID: \"6710ee7b-cbd5-4791-95c0-2c31b54e66b2\") " pod="openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn"
Jun 03 16:40:53.268599 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:53.268559    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn"
Jun 03 16:40:53.372067 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:53.371707    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-6577596cd5-kqzpv" event={"ID":"e25b10b0-b604-43a8-9d61-9ef957201a7e","Type":"ContainerStarted","Data":"f0fa2d19e5f0347e833af6fd61d8f7f0c0c9bbcfdbf5f6b9b23ce10fd6163709"}
Jun 03 16:40:53.373711 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:53.373679    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-5f8mp" event={"ID":"c2f6ac50-09f1-4f61-8160-530a2102718b","Type":"ContainerStarted","Data":"05163859d9f0fada7b7588688251939702615913265544aa1e5865a279e472c3"}
Jun 03 16:40:53.400099 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:53.400076    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn"]
Jun 03 16:40:53.402412 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:40:53.402385    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6710ee7b_cbd5_4791_95c0_2c31b54e66b2.slice/crio-cbc160420cc9aa3d8fb3e91cb1c924e9ce5319b00ca7e686bb9190e428069906 WatchSource:0}: Error finding container cbc160420cc9aa3d8fb3e91cb1c924e9ce5319b00ca7e686bb9190e428069906: Status 404 returned error can't find the container with id cbc160420cc9aa3d8fb3e91cb1c924e9ce5319b00ca7e686bb9190e428069906
Jun 03 16:40:54.384712 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:54.384613    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn" event={"ID":"6710ee7b-cbd5-4791-95c0-2c31b54e66b2","Type":"ContainerStarted","Data":"cbc160420cc9aa3d8fb3e91cb1c924e9ce5319b00ca7e686bb9190e428069906"}
Jun 03 16:40:58.398652 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:58.398609    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-6577596cd5-kqzpv" event={"ID":"e25b10b0-b604-43a8-9d61-9ef957201a7e","Type":"ContainerStarted","Data":"82bb6f7befa062f530ccf0de3090c05d6354fd4dfd697adad3c82893e8351d49"}
Jun 03 16:40:58.399189 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:58.398890    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-6577596cd5-kqzpv"
Jun 03 16:40:58.400035 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:58.400009    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-5f8mp" event={"ID":"c2f6ac50-09f1-4f61-8160-530a2102718b","Type":"ContainerStarted","Data":"e50fe825f46dba60ad43b25672b2e6d786a9fd448f2ee6f60a3b839030e69ccf"}
Jun 03 16:40:58.400181 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:58.400152    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-76588b95-5f8mp"
Jun 03 16:40:58.401246 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:58.401217    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn" event={"ID":"6710ee7b-cbd5-4791-95c0-2c31b54e66b2","Type":"ContainerStarted","Data":"e4eeb0cb0ab40cc3e96df204cf1e9ef79751c88f2fc77f39003e65275fb7c1a8"}
Jun 03 16:40:58.415303 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:58.415260    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/model-serving-api-6577596cd5-kqzpv" podStartSLOduration=1.508738664 podStartE2EDuration="6.415247713s" podCreationTimestamp="2026-06-03 16:40:52 +0000 UTC" firstStartedPulling="2026-06-03 16:40:53.005755941 +0000 UTC m=+422.536275507" lastFinishedPulling="2026-06-03 16:40:57.912264992 +0000 UTC m=+427.442784556" observedRunningTime="2026-06-03 16:40:58.414171636 +0000 UTC m=+427.944691230" watchObservedRunningTime="2026-06-03 16:40:58.415247713 +0000 UTC m=+427.945767299"
Jun 03 16:40:58.428495 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:58.428451    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-598f4f7759-pv7cn" podStartSLOduration=1.914394024 podStartE2EDuration="6.428441016s" podCreationTimestamp="2026-06-03 16:40:52 +0000 UTC" firstStartedPulling="2026-06-03 16:40:53.40431242 +0000 UTC m=+422.934831981" lastFinishedPulling="2026-06-03 16:40:57.91835941 +0000 UTC m=+427.448878973" observedRunningTime="2026-06-03 16:40:58.427750198 +0000 UTC m=+427.958269782" watchObservedRunningTime="2026-06-03 16:40:58.428441016 +0000 UTC m=+427.958960599"
Jun 03 16:40:58.444860 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:40:58.444816    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-76588b95-5f8mp" podStartSLOduration=2.469231872 podStartE2EDuration="7.44477996s" podCreationTimestamp="2026-06-03 16:40:51 +0000 UTC" firstStartedPulling="2026-06-03 16:40:52.932304007 +0000 UTC m=+422.462823588" lastFinishedPulling="2026-06-03 16:40:57.907852107 +0000 UTC m=+427.438371676" observedRunningTime="2026-06-03 16:40:58.443848767 +0000 UTC m=+427.974368377" watchObservedRunningTime="2026-06-03 16:40:58.44477996 +0000 UTC m=+427.975299544"
Jun 03 16:41:02.473441 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.473406    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv"]
Jun 03 16:41:02.476153 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.476138    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv"
Jun 03 16:41:02.479251 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.479223    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-n7mdv"]
Jun 03 16:41:02.479821 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.479781    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jun 03 16:41:02.479940 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.479896    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-m95xm\""
Jun 03 16:41:02.482262 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.482239    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-n7mdv"
Jun 03 16:41:02.485882 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.485861    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jun 03 16:41:02.485989 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.485946    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-x2p2z\""
Jun 03 16:41:02.488512 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.488493    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv"]
Jun 03 16:41:02.496980 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.496957    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-n7mdv"]
Jun 03 16:41:02.589357 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.589325    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fnlb\" (UniqueName: \"kubernetes.io/projected/ad7675de-c2c2-42c6-bb89-19fc317cd696-kube-api-access-5fnlb\") pod \"kserve-controller-manager-7c99ffcdcd-pswtv\" (UID: \"ad7675de-c2c2-42c6-bb89-19fc317cd696\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv"
Jun 03 16:41:02.589524 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.589386    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad7675de-c2c2-42c6-bb89-19fc317cd696-cert\") pod \"kserve-controller-manager-7c99ffcdcd-pswtv\" (UID: \"ad7675de-c2c2-42c6-bb89-19fc317cd696\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv"
Jun 03 16:41:02.589524 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.589465    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49a57957-b12d-4a75-979b-814b7f656492-cert\") pod \"llmisvc-controller-manager-c6b9847d7-n7mdv\" (UID: \"49a57957-b12d-4a75-979b-814b7f656492\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-n7mdv"
Jun 03 16:41:02.589524 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.589507    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbjtp\" (UniqueName: \"kubernetes.io/projected/49a57957-b12d-4a75-979b-814b7f656492-kube-api-access-bbjtp\") pod \"llmisvc-controller-manager-c6b9847d7-n7mdv\" (UID: \"49a57957-b12d-4a75-979b-814b7f656492\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-n7mdv"
Jun 03 16:41:02.690903 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.690863    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5fnlb\" (UniqueName: \"kubernetes.io/projected/ad7675de-c2c2-42c6-bb89-19fc317cd696-kube-api-access-5fnlb\") pod \"kserve-controller-manager-7c99ffcdcd-pswtv\" (UID: \"ad7675de-c2c2-42c6-bb89-19fc317cd696\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv"
Jun 03 16:41:02.691098 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.690948    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad7675de-c2c2-42c6-bb89-19fc317cd696-cert\") pod \"kserve-controller-manager-7c99ffcdcd-pswtv\" (UID: \"ad7675de-c2c2-42c6-bb89-19fc317cd696\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv"
Jun 03 16:41:02.691098 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.690995    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49a57957-b12d-4a75-979b-814b7f656492-cert\") pod \"llmisvc-controller-manager-c6b9847d7-n7mdv\" (UID: \"49a57957-b12d-4a75-979b-814b7f656492\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-n7mdv"
Jun 03 16:41:02.691098 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.691028    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bbjtp\" (UniqueName: \"kubernetes.io/projected/49a57957-b12d-4a75-979b-814b7f656492-kube-api-access-bbjtp\") pod \"llmisvc-controller-manager-c6b9847d7-n7mdv\" (UID: \"49a57957-b12d-4a75-979b-814b7f656492\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-n7mdv"
Jun 03 16:41:02.691261 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:41:02.691166    2569 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jun 03 16:41:02.691313 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:41:02.691264    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad7675de-c2c2-42c6-bb89-19fc317cd696-cert podName:ad7675de-c2c2-42c6-bb89-19fc317cd696 nodeName:}" failed. No retries permitted until 2026-06-03 16:41:03.19123969 +0000 UTC m=+432.721759264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ad7675de-c2c2-42c6-bb89-19fc317cd696-cert") pod "kserve-controller-manager-7c99ffcdcd-pswtv" (UID: "ad7675de-c2c2-42c6-bb89-19fc317cd696") : secret "kserve-webhook-server-cert" not found
Jun 03 16:41:02.693783 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.693759    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49a57957-b12d-4a75-979b-814b7f656492-cert\") pod \"llmisvc-controller-manager-c6b9847d7-n7mdv\" (UID: \"49a57957-b12d-4a75-979b-814b7f656492\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-n7mdv"
Jun 03 16:41:02.708290 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.708256    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fnlb\" (UniqueName: \"kubernetes.io/projected/ad7675de-c2c2-42c6-bb89-19fc317cd696-kube-api-access-5fnlb\") pod \"kserve-controller-manager-7c99ffcdcd-pswtv\" (UID: \"ad7675de-c2c2-42c6-bb89-19fc317cd696\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv"
Jun 03 16:41:02.715310 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.715288    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbjtp\" (UniqueName: \"kubernetes.io/projected/49a57957-b12d-4a75-979b-814b7f656492-kube-api-access-bbjtp\") pod \"llmisvc-controller-manager-c6b9847d7-n7mdv\" (UID: \"49a57957-b12d-4a75-979b-814b7f656492\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-n7mdv"
Jun 03 16:41:02.795251 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.795166    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-n7mdv"
Jun 03 16:41:02.918326 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:02.918294    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-n7mdv"]
Jun 03 16:41:02.921152 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:41:02.921127    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod49a57957_b12d_4a75_979b_814b7f656492.slice/crio-71a01cdfd1c4de18dd98e2d42bc3692b735892302023861661c0940f6f42b294 WatchSource:0}: Error finding container 71a01cdfd1c4de18dd98e2d42bc3692b735892302023861661c0940f6f42b294: Status 404 returned error can't find the container with id 71a01cdfd1c4de18dd98e2d42bc3692b735892302023861661c0940f6f42b294
Jun 03 16:41:03.195745 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:03.195703    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad7675de-c2c2-42c6-bb89-19fc317cd696-cert\") pod \"kserve-controller-manager-7c99ffcdcd-pswtv\" (UID: \"ad7675de-c2c2-42c6-bb89-19fc317cd696\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv"
Jun 03 16:41:03.200067 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:03.200034    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad7675de-c2c2-42c6-bb89-19fc317cd696-cert\") pod \"kserve-controller-manager-7c99ffcdcd-pswtv\" (UID: \"ad7675de-c2c2-42c6-bb89-19fc317cd696\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv"
Jun 03 16:41:03.386920 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:03.386886    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv"
Jun 03 16:41:03.416403 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:03.416372    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-n7mdv" event={"ID":"49a57957-b12d-4a75-979b-814b7f656492","Type":"ContainerStarted","Data":"71a01cdfd1c4de18dd98e2d42bc3692b735892302023861661c0940f6f42b294"}
Jun 03 16:41:03.513546 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:03.513520    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv"]
Jun 03 16:41:03.516323 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:41:03.516296    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad7675de_c2c2_42c6_bb89_19fc317cd696.slice/crio-d94f78b5d60fb27b42f9abee9bee1df4da2e7c7666516d8ca53d3e9f6f4674ff WatchSource:0}: Error finding container d94f78b5d60fb27b42f9abee9bee1df4da2e7c7666516d8ca53d3e9f6f4674ff: Status 404 returned error can't find the container with id d94f78b5d60fb27b42f9abee9bee1df4da2e7c7666516d8ca53d3e9f6f4674ff
Jun 03 16:41:04.420964 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:04.420929    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv" event={"ID":"ad7675de-c2c2-42c6-bb89-19fc317cd696","Type":"ContainerStarted","Data":"d94f78b5d60fb27b42f9abee9bee1df4da2e7c7666516d8ca53d3e9f6f4674ff"}
Jun 03 16:41:07.432942 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:07.432901    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-n7mdv" event={"ID":"49a57957-b12d-4a75-979b-814b7f656492","Type":"ContainerStarted","Data":"0849b92fbac870249ff358686b8ebde739d8a76d486403a4e9bfd88af110c794"}
Jun 03 16:41:07.433406 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:07.433033    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-n7mdv"
Jun 03 16:41:07.434365 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:07.434334    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv" event={"ID":"ad7675de-c2c2-42c6-bb89-19fc317cd696","Type":"ContainerStarted","Data":"8730f5c3fad6483028e59713b80d8a981d5b915d6a288e9b2119486a996f7263"}
Jun 03 16:41:07.434510 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:07.434446    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv"
Jun 03 16:41:07.477619 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:07.477576    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-n7mdv" podStartSLOduration=1.499190326 podStartE2EDuration="5.477567187s" podCreationTimestamp="2026-06-03 16:41:02 +0000 UTC" firstStartedPulling="2026-06-03 16:41:02.922494906 +0000 UTC m=+432.453014466" lastFinishedPulling="2026-06-03 16:41:06.900871761 +0000 UTC m=+436.431391327" observedRunningTime="2026-06-03 16:41:07.475987581 +0000 UTC m=+437.006507165" watchObservedRunningTime="2026-06-03 16:41:07.477567187 +0000 UTC m=+437.008086769"
Jun 03 16:41:07.539086 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:07.539042    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv" podStartSLOduration=2.154015285 podStartE2EDuration="5.53902778s" podCreationTimestamp="2026-06-03 16:41:02 +0000 UTC" firstStartedPulling="2026-06-03 16:41:03.518056979 +0000 UTC m=+433.048576544" lastFinishedPulling="2026-06-03 16:41:06.903069477 +0000 UTC m=+436.433589039" observedRunningTime="2026-06-03 16:41:07.537154454 +0000 UTC m=+437.067674048" watchObservedRunningTime="2026-06-03 16:41:07.53902778 +0000 UTC m=+437.069547362"
Jun 03 16:41:09.405975 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:09.405948    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-76588b95-5f8mp"
Jun 03 16:41:09.408205 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:09.408184    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-6577596cd5-kqzpv"
Jun 03 16:41:11.000046 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:11.000010    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-8twzk"]
Jun 03 16:41:11.005724 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:11.005701    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-8twzk"
Jun 03 16:41:11.008754 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:11.008734    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-qstcd\""
Jun 03 16:41:11.008952 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:11.008930    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\""
Jun 03 16:41:11.009253 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:11.009238    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\""
Jun 03 16:41:11.021640 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:11.021611    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-8twzk"]
Jun 03 16:41:11.158183 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:11.158141    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/cf8a3abf-1c8a-4e88-bd3f-2023d0d155f6-operator-config\") pod \"servicemesh-operator3-55f49c5f94-8twzk\" (UID: \"cf8a3abf-1c8a-4e88-bd3f-2023d0d155f6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-8twzk"
Jun 03 16:41:11.158346 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:11.158260    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr8wv\" (UniqueName: \"kubernetes.io/projected/cf8a3abf-1c8a-4e88-bd3f-2023d0d155f6-kube-api-access-hr8wv\") pod \"servicemesh-operator3-55f49c5f94-8twzk\" (UID: \"cf8a3abf-1c8a-4e88-bd3f-2023d0d155f6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-8twzk"
Jun 03 16:41:11.259477 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:11.259393    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/cf8a3abf-1c8a-4e88-bd3f-2023d0d155f6-operator-config\") pod \"servicemesh-operator3-55f49c5f94-8twzk\" (UID: \"cf8a3abf-1c8a-4e88-bd3f-2023d0d155f6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-8twzk"
Jun 03 16:41:11.259477 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:11.259448    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hr8wv\" (UniqueName: \"kubernetes.io/projected/cf8a3abf-1c8a-4e88-bd3f-2023d0d155f6-kube-api-access-hr8wv\") pod \"servicemesh-operator3-55f49c5f94-8twzk\" (UID: \"cf8a3abf-1c8a-4e88-bd3f-2023d0d155f6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-8twzk"
Jun 03 16:41:11.261961 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:11.261936    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/cf8a3abf-1c8a-4e88-bd3f-2023d0d155f6-operator-config\") pod \"servicemesh-operator3-55f49c5f94-8twzk\" (UID: \"cf8a3abf-1c8a-4e88-bd3f-2023d0d155f6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-8twzk"
Jun 03 16:41:11.268084 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:11.268064    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr8wv\" (UniqueName: \"kubernetes.io/projected/cf8a3abf-1c8a-4e88-bd3f-2023d0d155f6-kube-api-access-hr8wv\") pod \"servicemesh-operator3-55f49c5f94-8twzk\" (UID: \"cf8a3abf-1c8a-4e88-bd3f-2023d0d155f6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-8twzk"
Jun 03 16:41:11.315905 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:11.315883    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-8twzk"
Jun 03 16:41:11.446005 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:11.445978    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-8twzk"]
Jun 03 16:41:11.448264 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:41:11.448230    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf8a3abf_1c8a_4e88_bd3f_2023d0d155f6.slice/crio-a3e50145a4e054bf001177dededaa2cbe0e9ec79da74ff7b6691681ac112efc6 WatchSource:0}: Error finding container a3e50145a4e054bf001177dededaa2cbe0e9ec79da74ff7b6691681ac112efc6: Status 404 returned error can't find the container with id a3e50145a4e054bf001177dededaa2cbe0e9ec79da74ff7b6691681ac112efc6
Jun 03 16:41:12.454438 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:12.454395    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-8twzk" event={"ID":"cf8a3abf-1c8a-4e88-bd3f-2023d0d155f6","Type":"ContainerStarted","Data":"a3e50145a4e054bf001177dededaa2cbe0e9ec79da74ff7b6691681ac112efc6"}
Jun 03 16:41:15.466149 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:15.466113    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-8twzk" event={"ID":"cf8a3abf-1c8a-4e88-bd3f-2023d0d155f6","Type":"ContainerStarted","Data":"1c0e5521cd2fbca73001ce71e32f98dff579c50d0c01bc90e3ae6ab8463d36e5"}
Jun 03 16:41:15.466591 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:15.466227    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-8twzk"
Jun 03 16:41:15.489875 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:15.489827    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-8twzk" podStartSLOduration=2.5151841470000003 podStartE2EDuration="5.489812885s" podCreationTimestamp="2026-06-03 16:41:10 +0000 UTC" firstStartedPulling="2026-06-03 16:41:11.451346542 +0000 UTC m=+440.981866117" lastFinishedPulling="2026-06-03 16:41:14.425975294 +0000 UTC m=+443.956494855" observedRunningTime="2026-06-03 16:41:15.488626013 +0000 UTC m=+445.019145599" watchObservedRunningTime="2026-06-03 16:41:15.489812885 +0000 UTC m=+445.020332471"
Jun 03 16:41:26.471442 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.471412    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-8twzk"
Jun 03 16:41:26.613982 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.613948    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"]
Jun 03 16:41:26.621716 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.621695    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.625532 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.625508    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jun 03 16:41:26.625762 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.625506    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jun 03 16:41:26.625972 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.625530    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jun 03 16:41:26.626057 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.625646    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jun 03 16:41:26.626057 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.625770    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rz6t7\""
Jun 03 16:41:26.627643 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.627618    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"]
Jun 03 16:41:26.773548 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.773455    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.773548 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.773494    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-istio-token\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.773548 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.773526    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-local-certs\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.773784 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.773570    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-cacerts\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.773784 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.773661    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.773784 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.773702    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.773784 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.773720    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lvr5\" (UniqueName: \"kubernetes.io/projected/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-kube-api-access-9lvr5\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.874907 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.874865    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.874907 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.874901    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-istio-token\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.875161 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.874925    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-local-certs\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.875161 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.874968    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-cacerts\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.875161 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.875007    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.875161 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.875035    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.875161 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.875058    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9lvr5\" (UniqueName: \"kubernetes.io/projected/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-kube-api-access-9lvr5\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.876017 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.875989    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.877527 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.877501    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.877527 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.877517    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.877713 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.877599    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-cacerts\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.877830 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.877812    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-local-certs\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.883162 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.883130    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-istio-token\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.883645 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.883622    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lvr5\" (UniqueName: \"kubernetes.io/projected/6465d0cd-83f6-4b71-b5fa-43e4017ab0e5-kube-api-access-9lvr5\") pod \"istiod-openshift-gateway-798958bb55-rs8sk\" (UID: \"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:26.932751 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:26.932705    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:27.066192 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:27.066160    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"]
Jun 03 16:41:27.068225 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:41:27.068193    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6465d0cd_83f6_4b71_b5fa_43e4017ab0e5.slice/crio-53b7fd4b8d5a016a313f96c1cd8842a11ea6242e914d31adadb88a50bc1277e3 WatchSource:0}: Error finding container 53b7fd4b8d5a016a313f96c1cd8842a11ea6242e914d31adadb88a50bc1277e3: Status 404 returned error can't find the container with id 53b7fd4b8d5a016a313f96c1cd8842a11ea6242e914d31adadb88a50bc1277e3
Jun 03 16:41:27.506288 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:27.506187    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk" event={"ID":"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5","Type":"ContainerStarted","Data":"53b7fd4b8d5a016a313f96c1cd8842a11ea6242e914d31adadb88a50bc1277e3"}
Jun 03 16:41:29.968771 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:29.968729    2569 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jun 03 16:41:29.969120 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:29.968817    2569 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jun 03 16:41:30.519662 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:30.519627    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk" event={"ID":"6465d0cd-83f6-4b71-b5fa-43e4017ab0e5","Type":"ContainerStarted","Data":"7b66dd71f4578cd522bf2f32c2a0f5c4df699194a409263116dcffee156e352d"}
Jun 03 16:41:30.519880 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:30.519730    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:30.544062 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:30.542082    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk" podStartSLOduration=1.6438984890000001 podStartE2EDuration="4.542068197s" podCreationTimestamp="2026-06-03 16:41:26 +0000 UTC" firstStartedPulling="2026-06-03 16:41:27.070307741 +0000 UTC m=+456.600827301" lastFinishedPulling="2026-06-03 16:41:29.968477448 +0000 UTC m=+459.498997009" observedRunningTime="2026-06-03 16:41:30.54147412 +0000 UTC m=+460.071993704" watchObservedRunningTime="2026-06-03 16:41:30.542068197 +0000 UTC m=+460.072587781"
Jun 03 16:41:31.524815 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:31.524761    2569 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-rs8sk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jun 03 16:41:31.525194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:31.524856    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk" podUID="6465d0cd-83f6-4b71-b5fa-43e4017ab0e5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 03 16:41:34.524706 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:34.524669    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-rs8sk"
Jun 03 16:41:34.972813 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:34.972758    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-99f686c75-zzx5m"]
Jun 03 16:41:34.978263 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:34.978238    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-99f686c75-zzx5m"
Jun 03 16:41:34.982186 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:34.982160    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\""
Jun 03 16:41:34.982340 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:34.982198    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-d56zr\""
Jun 03 16:41:34.983565 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:34.983542    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-99f686c75-zzx5m"]
Jun 03 16:41:35.045688 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:35.045653    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrglv\" (UniqueName: \"kubernetes.io/projected/93668359-87fa-4669-b741-9f66c4769b5d-kube-api-access-hrglv\") pod \"dashboard-redirect-99f686c75-zzx5m\" (UID: \"93668359-87fa-4669-b741-9f66c4769b5d\") " pod="opendatahub/dashboard-redirect-99f686c75-zzx5m"
Jun 03 16:41:35.045863 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:35.045715    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/93668359-87fa-4669-b741-9f66c4769b5d-redirect-config\") pod \"dashboard-redirect-99f686c75-zzx5m\" (UID: \"93668359-87fa-4669-b741-9f66c4769b5d\") " pod="opendatahub/dashboard-redirect-99f686c75-zzx5m"
Jun 03 16:41:35.146210 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:35.146158    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/93668359-87fa-4669-b741-9f66c4769b5d-redirect-config\") pod \"dashboard-redirect-99f686c75-zzx5m\" (UID: \"93668359-87fa-4669-b741-9f66c4769b5d\") " pod="opendatahub/dashboard-redirect-99f686c75-zzx5m"
Jun 03 16:41:35.146397 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:35.146266    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hrglv\" (UniqueName: \"kubernetes.io/projected/93668359-87fa-4669-b741-9f66c4769b5d-kube-api-access-hrglv\") pod \"dashboard-redirect-99f686c75-zzx5m\" (UID: \"93668359-87fa-4669-b741-9f66c4769b5d\") " pod="opendatahub/dashboard-redirect-99f686c75-zzx5m"
Jun 03 16:41:35.146985 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:35.146963    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/93668359-87fa-4669-b741-9f66c4769b5d-redirect-config\") pod \"dashboard-redirect-99f686c75-zzx5m\" (UID: \"93668359-87fa-4669-b741-9f66c4769b5d\") " pod="opendatahub/dashboard-redirect-99f686c75-zzx5m"
Jun 03 16:41:35.153820 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:35.153762    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrglv\" (UniqueName: \"kubernetes.io/projected/93668359-87fa-4669-b741-9f66c4769b5d-kube-api-access-hrglv\") pod \"dashboard-redirect-99f686c75-zzx5m\" (UID: \"93668359-87fa-4669-b741-9f66c4769b5d\") " pod="opendatahub/dashboard-redirect-99f686c75-zzx5m"
Jun 03 16:41:35.291867 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:35.291757    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-99f686c75-zzx5m"
Jun 03 16:41:35.421714 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:35.421688    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-99f686c75-zzx5m"]
Jun 03 16:41:35.423649 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:41:35.423617    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93668359_87fa_4669_b741_9f66c4769b5d.slice/crio-5dc39cb315fb5e6871017677e0f0c0353fd423d065c62084c1bbdf20bbd214af WatchSource:0}: Error finding container 5dc39cb315fb5e6871017677e0f0c0353fd423d065c62084c1bbdf20bbd214af: Status 404 returned error can't find the container with id 5dc39cb315fb5e6871017677e0f0c0353fd423d065c62084c1bbdf20bbd214af
Jun 03 16:41:35.536402 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:35.536334    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-99f686c75-zzx5m" event={"ID":"93668359-87fa-4669-b741-9f66c4769b5d","Type":"ContainerStarted","Data":"5dc39cb315fb5e6871017677e0f0c0353fd423d065c62084c1bbdf20bbd214af"}
Jun 03 16:41:38.440021 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:38.439981    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-n7mdv"
Jun 03 16:41:38.444574 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:38.444526    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-pswtv"
Jun 03 16:41:40.557843 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:40.557813    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-99f686c75-zzx5m" event={"ID":"93668359-87fa-4669-b741-9f66c4769b5d","Type":"ContainerStarted","Data":"9d794fc7be22b4369eb6c4126f0a7f01cb301e9c1ba5daab0d34ede9f0871fee"}
Jun 03 16:41:40.574734 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:41:40.574685    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-99f686c75-zzx5m" podStartSLOduration=1.856246032 podStartE2EDuration="6.574671544s" podCreationTimestamp="2026-06-03 16:41:34 +0000 UTC" firstStartedPulling="2026-06-03 16:41:35.425006561 +0000 UTC m=+464.955526126" lastFinishedPulling="2026-06-03 16:41:40.143432068 +0000 UTC m=+469.673951638" observedRunningTime="2026-06-03 16:41:40.573469886 +0000 UTC m=+470.103989471" watchObservedRunningTime="2026-06-03 16:41:40.574671544 +0000 UTC m=+470.105191127"
Jun 03 16:42:23.528171 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:23.528123    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx"]
Jun 03 16:42:23.553599 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:23.553562    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx"]
Jun 03 16:42:23.553741 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:23.553711    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx"
Jun 03 16:42:23.556547 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:23.556515    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jun 03 16:42:23.557904 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:23.557884    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jun 03 16:42:23.557998 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:23.557889    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-zw9bf\""
Jun 03 16:42:23.645284 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:23.645246    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf52m\" (UniqueName: \"kubernetes.io/projected/1ef2682f-400c-4fd6-8176-f8d65d317c66-kube-api-access-tf52m\") pod \"limitador-operator-controller-manager-85c4996f8c-d5pxx\" (UID: \"1ef2682f-400c-4fd6-8176-f8d65d317c66\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx"
Jun 03 16:42:23.746718 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:23.746679    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tf52m\" (UniqueName: \"kubernetes.io/projected/1ef2682f-400c-4fd6-8176-f8d65d317c66-kube-api-access-tf52m\") pod \"limitador-operator-controller-manager-85c4996f8c-d5pxx\" (UID: \"1ef2682f-400c-4fd6-8176-f8d65d317c66\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx"
Jun 03 16:42:23.755785 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:23.755760    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf52m\" (UniqueName: \"kubernetes.io/projected/1ef2682f-400c-4fd6-8176-f8d65d317c66-kube-api-access-tf52m\") pod \"limitador-operator-controller-manager-85c4996f8c-d5pxx\" (UID: \"1ef2682f-400c-4fd6-8176-f8d65d317c66\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx"
Jun 03 16:42:23.863870 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:23.863837    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx"
Jun 03 16:42:23.992315 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:23.992281    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx"]
Jun 03 16:42:23.994712 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:42:23.994683    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ef2682f_400c_4fd6_8176_f8d65d317c66.slice/crio-334e1f16718ccd1c4cfbf423021459cd29c384ccde2a3a6c85488a4687a5730e WatchSource:0}: Error finding container 334e1f16718ccd1c4cfbf423021459cd29c384ccde2a3a6c85488a4687a5730e: Status 404 returned error can't find the container with id 334e1f16718ccd1c4cfbf423021459cd29c384ccde2a3a6c85488a4687a5730e
Jun 03 16:42:24.693809 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:24.693755    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx" event={"ID":"1ef2682f-400c-4fd6-8176-f8d65d317c66","Type":"ContainerStarted","Data":"334e1f16718ccd1c4cfbf423021459cd29c384ccde2a3a6c85488a4687a5730e"}
Jun 03 16:42:26.701689 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:26.701658    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx" event={"ID":"1ef2682f-400c-4fd6-8176-f8d65d317c66","Type":"ContainerStarted","Data":"9df83af9d53819e151223f3f2a36067f6da7427eb3f905216abaeb05e5f60689"}
Jun 03 16:42:26.702150 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:26.701771    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx"
Jun 03 16:42:26.723551 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:26.723501    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx" podStartSLOduration=1.682720971 podStartE2EDuration="3.723486038s" podCreationTimestamp="2026-06-03 16:42:23 +0000 UTC" firstStartedPulling="2026-06-03 16:42:23.996532571 +0000 UTC m=+513.527052135" lastFinishedPulling="2026-06-03 16:42:26.037297642 +0000 UTC m=+515.567817202" observedRunningTime="2026-06-03 16:42:26.7213614 +0000 UTC m=+516.251880983" watchObservedRunningTime="2026-06-03 16:42:26.723486038 +0000 UTC m=+516.254005620"
Jun 03 16:42:37.706940 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:37.706905    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx"
Jun 03 16:42:50.717194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:50.717158    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw"]
Jun 03 16:42:50.723589 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:50.723559    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw"
Jun 03 16:42:50.726684 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:50.726665    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qqh2p\""
Jun 03 16:42:50.734630 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:50.734608    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw"]
Jun 03 16:42:50.751004 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:50.750977    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqr8v\" (UniqueName: \"kubernetes.io/projected/96590e1e-746e-467a-b167-306ec67bfeac-kube-api-access-zqr8v\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw\" (UID: \"96590e1e-746e-467a-b167-306ec67bfeac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw"
Jun 03 16:42:50.751148 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:50.751043    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/96590e1e-746e-467a-b167-306ec67bfeac-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw\" (UID: \"96590e1e-746e-467a-b167-306ec67bfeac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw"
Jun 03 16:42:50.851476 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:50.851438    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zqr8v\" (UniqueName: \"kubernetes.io/projected/96590e1e-746e-467a-b167-306ec67bfeac-kube-api-access-zqr8v\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw\" (UID: \"96590e1e-746e-467a-b167-306ec67bfeac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw"
Jun 03 16:42:50.851646 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:50.851504    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/96590e1e-746e-467a-b167-306ec67bfeac-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw\" (UID: \"96590e1e-746e-467a-b167-306ec67bfeac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw"
Jun 03 16:42:50.851878 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:50.851861    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/96590e1e-746e-467a-b167-306ec67bfeac-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw\" (UID: \"96590e1e-746e-467a-b167-306ec67bfeac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw"
Jun 03 16:42:50.860100 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:50.860076    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqr8v\" (UniqueName: \"kubernetes.io/projected/96590e1e-746e-467a-b167-306ec67bfeac-kube-api-access-zqr8v\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw\" (UID: \"96590e1e-746e-467a-b167-306ec67bfeac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw"
Jun 03 16:42:51.037423 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.037351    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qqh2p\""
Jun 03 16:42:51.044593 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.044565    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw"
Jun 03 16:42:51.174196 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.174169    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw"]
Jun 03 16:42:51.175919 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:42:51.175890    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96590e1e_746e_467a_b167_306ec67bfeac.slice/crio-e85b7f6ba5a93d3a76b77b2f859617cd0d4842554cefbbd082f3f45af5f2defe WatchSource:0}: Error finding container e85b7f6ba5a93d3a76b77b2f859617cd0d4842554cefbbd082f3f45af5f2defe: Status 404 returned error can't find the container with id e85b7f6ba5a93d3a76b77b2f859617cd0d4842554cefbbd082f3f45af5f2defe
Jun 03 16:42:51.495370 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.495336    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw"]
Jun 03 16:42:51.499834 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.499790    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw"]
Jun 03 16:42:51.519367 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.519338    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx"]
Jun 03 16:42:51.519835 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.519577    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx" podUID="1ef2682f-400c-4fd6-8176-f8d65d317c66" containerName="manager" containerID="cri-o://9df83af9d53819e151223f3f2a36067f6da7427eb3f905216abaeb05e5f60689" gracePeriod=2
Jun 03 16:42:51.531277 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.531216    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx"]
Jun 03 16:42:51.541434 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.541407    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-mrqnn"]
Jun 03 16:42:51.541724 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.541712    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ef2682f-400c-4fd6-8176-f8d65d317c66" containerName="manager"
Jun 03 16:42:51.541772 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.541726    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ef2682f-400c-4fd6-8176-f8d65d317c66" containerName="manager"
Jun 03 16:42:51.541819 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.541787    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="1ef2682f-400c-4fd6-8176-f8d65d317c66" containerName="manager"
Jun 03 16:42:51.545275 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.545256    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-mrqnn"
Jun 03 16:42:51.550874 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.550785    2569 status_manager.go:895] "Failed to get status for pod" podUID="1ef2682f-400c-4fd6-8176-f8d65d317c66" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx" err="pods \"limitador-operator-controller-manager-85c4996f8c-d5pxx\" is forbidden: User \"system:node:ip-10-0-140-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-123.ec2.internal' and this object"
Jun 03 16:42:51.557124 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.557097    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-mrqnn"]
Jun 03 16:42:51.558219 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.558193    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2ct6\" (UniqueName: \"kubernetes.io/projected/1ccc8c72-c970-40cc-8665-96a0a2195779-kube-api-access-d2ct6\") pod \"limitador-operator-controller-manager-85c4996f8c-mrqnn\" (UID: \"1ccc8c72-c970-40cc-8665-96a0a2195779\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-mrqnn"
Jun 03 16:42:51.659674 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.659637    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d2ct6\" (UniqueName: \"kubernetes.io/projected/1ccc8c72-c970-40cc-8665-96a0a2195779-kube-api-access-d2ct6\") pod \"limitador-operator-controller-manager-85c4996f8c-mrqnn\" (UID: \"1ccc8c72-c970-40cc-8665-96a0a2195779\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-mrqnn"
Jun 03 16:42:51.682287 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.682253    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2ct6\" (UniqueName: \"kubernetes.io/projected/1ccc8c72-c970-40cc-8665-96a0a2195779-kube-api-access-d2ct6\") pod \"limitador-operator-controller-manager-85c4996f8c-mrqnn\" (UID: \"1ccc8c72-c970-40cc-8665-96a0a2195779\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-mrqnn"
Jun 03 16:42:51.786259 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.786185    2569 generic.go:358] "Generic (PLEG): container finished" podID="1ef2682f-400c-4fd6-8176-f8d65d317c66" containerID="9df83af9d53819e151223f3f2a36067f6da7427eb3f905216abaeb05e5f60689" exitCode=0
Jun 03 16:42:51.903217 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.903190    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-mrqnn"
Jun 03 16:42:51.909061 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.909040    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx"
Jun 03 16:42:51.911425 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.911394    2569 status_manager.go:895] "Failed to get status for pod" podUID="1ef2682f-400c-4fd6-8176-f8d65d317c66" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx" err="pods \"limitador-operator-controller-manager-85c4996f8c-d5pxx\" is forbidden: User \"system:node:ip-10-0-140-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-123.ec2.internal' and this object"
Jun 03 16:42:51.962910 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.962875    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tf52m\" (UniqueName: \"kubernetes.io/projected/1ef2682f-400c-4fd6-8176-f8d65d317c66-kube-api-access-tf52m\") pod \"1ef2682f-400c-4fd6-8176-f8d65d317c66\" (UID: \"1ef2682f-400c-4fd6-8176-f8d65d317c66\") "
Jun 03 16:42:51.965383 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:51.965353    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ef2682f-400c-4fd6-8176-f8d65d317c66-kube-api-access-tf52m" (OuterVolumeSpecName: "kube-api-access-tf52m") pod "1ef2682f-400c-4fd6-8176-f8d65d317c66" (UID: "1ef2682f-400c-4fd6-8176-f8d65d317c66"). InnerVolumeSpecName "kube-api-access-tf52m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 16:42:52.063483 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:52.063454    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tf52m\" (UniqueName: \"kubernetes.io/projected/1ef2682f-400c-4fd6-8176-f8d65d317c66-kube-api-access-tf52m\") on node \"ip-10-0-140-123.ec2.internal\" DevicePath \"\""
Jun 03 16:42:52.214830 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:52.214788    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-mrqnn"]
Jun 03 16:42:52.216534 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:42:52.216505    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ccc8c72_c970_40cc_8665_96a0a2195779.slice/crio-f31cb1022d2f2db1603702fc7cc2aa1c967066286a11388a39478d7d74b360f2 WatchSource:0}: Error finding container f31cb1022d2f2db1603702fc7cc2aa1c967066286a11388a39478d7d74b360f2: Status 404 returned error can't find the container with id f31cb1022d2f2db1603702fc7cc2aa1c967066286a11388a39478d7d74b360f2
Jun 03 16:42:52.790833 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:52.790789    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx"
Jun 03 16:42:52.791282 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:52.790842    2569 scope.go:117] "RemoveContainer" containerID="9df83af9d53819e151223f3f2a36067f6da7427eb3f905216abaeb05e5f60689"
Jun 03 16:42:52.792210 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:52.792181    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-mrqnn" event={"ID":"1ccc8c72-c970-40cc-8665-96a0a2195779","Type":"ContainerStarted","Data":"a18b3a9e33442bafb8c0682e9f72519f9243597ccf2c9cba09539896427630aa"}
Jun 03 16:42:52.792308 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:52.792214    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-mrqnn" event={"ID":"1ccc8c72-c970-40cc-8665-96a0a2195779","Type":"ContainerStarted","Data":"f31cb1022d2f2db1603702fc7cc2aa1c967066286a11388a39478d7d74b360f2"}
Jun 03 16:42:52.792373 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:52.792334    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-mrqnn"
Jun 03 16:42:52.793346 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:52.793324    2569 status_manager.go:895] "Failed to get status for pod" podUID="1ef2682f-400c-4fd6-8176-f8d65d317c66" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx" err="pods \"limitador-operator-controller-manager-85c4996f8c-d5pxx\" is forbidden: User \"system:node:ip-10-0-140-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-123.ec2.internal' and this object"
Jun 03 16:42:52.810032 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:52.809995    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-mrqnn" podStartSLOduration=1.809983466 podStartE2EDuration="1.809983466s" podCreationTimestamp="2026-06-03 16:42:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:42:52.809019993 +0000 UTC m=+542.339539578" watchObservedRunningTime="2026-06-03 16:42:52.809983466 +0000 UTC m=+542.340503049"
Jun 03 16:42:52.811252 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:52.811228    2569 status_manager.go:895] "Failed to get status for pod" podUID="1ef2682f-400c-4fd6-8176-f8d65d317c66" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx" err="pods \"limitador-operator-controller-manager-85c4996f8c-d5pxx\" is forbidden: User \"system:node:ip-10-0-140-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-123.ec2.internal' and this object"
Jun 03 16:42:52.813255 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:52.813234    2569 status_manager.go:895] "Failed to get status for pod" podUID="1ef2682f-400c-4fd6-8176-f8d65d317c66" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-d5pxx" err="pods \"limitador-operator-controller-manager-85c4996f8c-d5pxx\" is forbidden: User \"system:node:ip-10-0-140-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-123.ec2.internal' and this object"
Jun 03 16:42:53.073512 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:53.073473    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ef2682f-400c-4fd6-8176-f8d65d317c66" path="/var/lib/kubelet/pods/1ef2682f-400c-4fd6-8176-f8d65d317c66/volumes"
Jun 03 16:42:55.804647 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:55.804591    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw" podUID="96590e1e-746e-467a-b167-306ec67bfeac" containerName="manager" containerID="cri-o://12361246873f0a2d9ea7e32fd1eb9ecd3a813ca2d933e3ece755e1f822818342" gracePeriod=2
Jun 03 16:42:56.036171 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:56.036148    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw"
Jun 03 16:42:56.038737 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:56.038709    2569 status_manager.go:895] "Failed to get status for pod" podUID="96590e1e-746e-467a-b167-306ec67bfeac" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw\" is forbidden: User \"system:node:ip-10-0-140-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-123.ec2.internal' and this object"
Jun 03 16:42:56.098631 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:56.098537    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqr8v\" (UniqueName: \"kubernetes.io/projected/96590e1e-746e-467a-b167-306ec67bfeac-kube-api-access-zqr8v\") pod \"96590e1e-746e-467a-b167-306ec67bfeac\" (UID: \"96590e1e-746e-467a-b167-306ec67bfeac\") "
Jun 03 16:42:56.098631 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:56.098587    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/96590e1e-746e-467a-b167-306ec67bfeac-extensions-socket-volume\") pod \"96590e1e-746e-467a-b167-306ec67bfeac\" (UID: \"96590e1e-746e-467a-b167-306ec67bfeac\") "
Jun 03 16:42:56.098929 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:56.098889    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96590e1e-746e-467a-b167-306ec67bfeac-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "96590e1e-746e-467a-b167-306ec67bfeac" (UID: "96590e1e-746e-467a-b167-306ec67bfeac"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 03 16:42:56.100651 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:56.100626    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96590e1e-746e-467a-b167-306ec67bfeac-kube-api-access-zqr8v" (OuterVolumeSpecName: "kube-api-access-zqr8v") pod "96590e1e-746e-467a-b167-306ec67bfeac" (UID: "96590e1e-746e-467a-b167-306ec67bfeac"). InnerVolumeSpecName "kube-api-access-zqr8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 16:42:56.199573 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:56.199537    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqr8v\" (UniqueName: \"kubernetes.io/projected/96590e1e-746e-467a-b167-306ec67bfeac-kube-api-access-zqr8v\") on node \"ip-10-0-140-123.ec2.internal\" DevicePath \"\""
Jun 03 16:42:56.199573 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:56.199567    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/96590e1e-746e-467a-b167-306ec67bfeac-extensions-socket-volume\") on node \"ip-10-0-140-123.ec2.internal\" DevicePath \"\""
Jun 03 16:42:56.811200 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:56.811168    2569 generic.go:358] "Generic (PLEG): container finished" podID="96590e1e-746e-467a-b167-306ec67bfeac" containerID="12361246873f0a2d9ea7e32fd1eb9ecd3a813ca2d933e3ece755e1f822818342" exitCode=2
Jun 03 16:42:56.811630 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:56.811236    2569 scope.go:117] "RemoveContainer" containerID="12361246873f0a2d9ea7e32fd1eb9ecd3a813ca2d933e3ece755e1f822818342"
Jun 03 16:42:56.811630 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:56.811233    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw"
Jun 03 16:42:56.813999 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:56.813966    2569 status_manager.go:895] "Failed to get status for pod" podUID="96590e1e-746e-467a-b167-306ec67bfeac" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw\" is forbidden: User \"system:node:ip-10-0-140-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-123.ec2.internal' and this object"
Jun 03 16:42:56.819465 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:56.819448    2569 scope.go:117] "RemoveContainer" containerID="12361246873f0a2d9ea7e32fd1eb9ecd3a813ca2d933e3ece755e1f822818342"
Jun 03 16:42:56.819717 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:42:56.819695    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12361246873f0a2d9ea7e32fd1eb9ecd3a813ca2d933e3ece755e1f822818342\": container with ID starting with 12361246873f0a2d9ea7e32fd1eb9ecd3a813ca2d933e3ece755e1f822818342 not found: ID does not exist" containerID="12361246873f0a2d9ea7e32fd1eb9ecd3a813ca2d933e3ece755e1f822818342"
Jun 03 16:42:56.819771 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:56.819726    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12361246873f0a2d9ea7e32fd1eb9ecd3a813ca2d933e3ece755e1f822818342"} err="failed to get container status \"12361246873f0a2d9ea7e32fd1eb9ecd3a813ca2d933e3ece755e1f822818342\": rpc error: code = NotFound desc = could not find container \"12361246873f0a2d9ea7e32fd1eb9ecd3a813ca2d933e3ece755e1f822818342\": container with ID starting with 12361246873f0a2d9ea7e32fd1eb9ecd3a813ca2d933e3ece755e1f822818342 not found: ID does not exist"
Jun 03 16:42:56.821813 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:56.821781    2569 status_manager.go:895] "Failed to get status for pod" podUID="96590e1e-746e-467a-b167-306ec67bfeac" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-b4zpw\" is forbidden: User \"system:node:ip-10-0-140-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-123.ec2.internal' and this object"
Jun 03 16:42:57.072651 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:42:57.072565    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96590e1e-746e-467a-b167-306ec67bfeac" path="/var/lib/kubelet/pods/96590e1e-746e-467a-b167-306ec67bfeac/volumes"
Jun 03 16:43:03.798386 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:03.798350    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-mrqnn"
Jun 03 16:43:23.717833 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:23.717786    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-49f8x"]
Jun 03 16:43:23.718221 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:23.718087    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96590e1e-746e-467a-b167-306ec67bfeac" containerName="manager"
Jun 03 16:43:23.718221 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:23.718097    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="96590e1e-746e-467a-b167-306ec67bfeac" containerName="manager"
Jun 03 16:43:23.718221 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:23.718160    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="96590e1e-746e-467a-b167-306ec67bfeac" containerName="manager"
Jun 03 16:43:23.723682 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:23.723661    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-49f8x"
Jun 03 16:43:23.726816 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:23.726770    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qqh2p\""
Jun 03 16:43:23.732912 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:23.732887    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-49f8x"]
Jun 03 16:43:23.837104 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:23.837077    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npcx6\" (UniqueName: \"kubernetes.io/projected/90a7e488-0ee4-486d-b84a-b2ca922314f7-kube-api-access-npcx6\") pod \"kuadrant-operator-controller-manager-55c7f4c975-49f8x\" (UID: \"90a7e488-0ee4-486d-b84a-b2ca922314f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-49f8x"
Jun 03 16:43:23.837259 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:23.837121    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90a7e488-0ee4-486d-b84a-b2ca922314f7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-49f8x\" (UID: \"90a7e488-0ee4-486d-b84a-b2ca922314f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-49f8x"
Jun 03 16:43:23.938225 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:23.938194    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-npcx6\" (UniqueName: \"kubernetes.io/projected/90a7e488-0ee4-486d-b84a-b2ca922314f7-kube-api-access-npcx6\") pod \"kuadrant-operator-controller-manager-55c7f4c975-49f8x\" (UID: \"90a7e488-0ee4-486d-b84a-b2ca922314f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-49f8x"
Jun 03 16:43:23.938398 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:23.938240    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90a7e488-0ee4-486d-b84a-b2ca922314f7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-49f8x\" (UID: \"90a7e488-0ee4-486d-b84a-b2ca922314f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-49f8x"
Jun 03 16:43:23.938589 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:23.938571    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/90a7e488-0ee4-486d-b84a-b2ca922314f7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-49f8x\" (UID: \"90a7e488-0ee4-486d-b84a-b2ca922314f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-49f8x"
Jun 03 16:43:23.947194 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:23.947172    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npcx6\" (UniqueName: \"kubernetes.io/projected/90a7e488-0ee4-486d-b84a-b2ca922314f7-kube-api-access-npcx6\") pod \"kuadrant-operator-controller-manager-55c7f4c975-49f8x\" (UID: \"90a7e488-0ee4-486d-b84a-b2ca922314f7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-49f8x"
Jun 03 16:43:24.034534 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:24.034449    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-49f8x"
Jun 03 16:43:24.158839 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:24.158769    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-49f8x"]
Jun 03 16:43:24.161306 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:43:24.161273    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90a7e488_0ee4_486d_b84a_b2ca922314f7.slice/crio-bed1cb4b9984064005f5674c5f7e3c189885eee4c8d09127637bfc3dd501f612 WatchSource:0}: Error finding container bed1cb4b9984064005f5674c5f7e3c189885eee4c8d09127637bfc3dd501f612: Status 404 returned error can't find the container with id bed1cb4b9984064005f5674c5f7e3c189885eee4c8d09127637bfc3dd501f612
Jun 03 16:43:24.916179 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:24.916144    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-49f8x" event={"ID":"90a7e488-0ee4-486d-b84a-b2ca922314f7","Type":"ContainerStarted","Data":"b91f2ce6ccc7cb84fcaf397dac15bf92619e378c2613abb03d27e738a582f125"}
Jun 03 16:43:24.916179 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:24.916180    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-49f8x" event={"ID":"90a7e488-0ee4-486d-b84a-b2ca922314f7","Type":"ContainerStarted","Data":"bed1cb4b9984064005f5674c5f7e3c189885eee4c8d09127637bfc3dd501f612"}
Jun 03 16:43:24.916614 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:24.916294    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-49f8x"
Jun 03 16:43:24.936895 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:24.936839    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-49f8x" podStartSLOduration=1.936816886 podStartE2EDuration="1.936816886s" podCreationTimestamp="2026-06-03 16:43:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:43:24.934678202 +0000 UTC m=+574.465197786" watchObservedRunningTime="2026-06-03 16:43:24.936816886 +0000 UTC m=+574.467336481"
Jun 03 16:43:35.921947 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:43:35.921915    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-49f8x"
Jun 03 16:44:40.293629 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:40.293595    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 16:44:40.296742 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:40.296725    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 03 16:44:40.299493 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:40.299471    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\""
Jun 03 16:44:40.299624 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:40.299522    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\""
Jun 03 16:44:40.299624 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:40.299522    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\""
Jun 03 16:44:40.300930 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:40.300913    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-p28sv\""
Jun 03 16:44:40.310998 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:40.310971    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 16:44:40.427973 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:40.427928    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhgvj\" (UniqueName: \"kubernetes.io/projected/eccb3201-6c8a-41ba-8774-5f9c4b6b0638-kube-api-access-nhgvj\") pod \"maas-keycloak-0\" (UID: \"eccb3201-6c8a-41ba-8774-5f9c4b6b0638\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 16:44:40.528989 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:40.528955    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nhgvj\" (UniqueName: \"kubernetes.io/projected/eccb3201-6c8a-41ba-8774-5f9c4b6b0638-kube-api-access-nhgvj\") pod \"maas-keycloak-0\" (UID: \"eccb3201-6c8a-41ba-8774-5f9c4b6b0638\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 16:44:40.540590 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:40.540563    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhgvj\" (UniqueName: \"kubernetes.io/projected/eccb3201-6c8a-41ba-8774-5f9c4b6b0638-kube-api-access-nhgvj\") pod \"maas-keycloak-0\" (UID: \"eccb3201-6c8a-41ba-8774-5f9c4b6b0638\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 16:44:40.612587 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:40.612549    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 03 16:44:40.737116 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:40.737092    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 16:44:40.738878 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:44:40.738845    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeccb3201_6c8a_41ba_8774_5f9c4b6b0638.slice/crio-49731b7337ba9be85721b60d3307d1b643f2868d54db0962202b5422b32991eb WatchSource:0}: Error finding container 49731b7337ba9be85721b60d3307d1b643f2868d54db0962202b5422b32991eb: Status 404 returned error can't find the container with id 49731b7337ba9be85721b60d3307d1b643f2868d54db0962202b5422b32991eb
Jun 03 16:44:41.162280 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:41.162241    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"eccb3201-6c8a-41ba-8774-5f9c4b6b0638","Type":"ContainerStarted","Data":"49731b7337ba9be85721b60d3307d1b643f2868d54db0962202b5422b32991eb"}
Jun 03 16:44:45.178860 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:45.178827    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"eccb3201-6c8a-41ba-8774-5f9c4b6b0638","Type":"ContainerStarted","Data":"2c28b71376005bd764e2b7f9f2a3a5c802e1fda4d693749baeccfdb7f5c85164"}
Jun 03 16:44:45.196007 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:45.195942    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=0.82166834 podStartE2EDuration="5.19592554s" podCreationTimestamp="2026-06-03 16:44:40 +0000 UTC" firstStartedPulling="2026-06-03 16:44:40.740074906 +0000 UTC m=+650.270594467" lastFinishedPulling="2026-06-03 16:44:45.114332101 +0000 UTC m=+654.644851667" observedRunningTime="2026-06-03 16:44:45.194370168 +0000 UTC m=+654.724889753" watchObservedRunningTime="2026-06-03 16:44:45.19592554 +0000 UTC m=+654.726445122"
Jun 03 16:44:45.613084 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:45.613033    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0"
Jun 03 16:44:45.614908 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:45.614871    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused"
Jun 03 16:44:46.613353 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:46.613281    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused"
Jun 03 16:44:47.613968 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:47.613913    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused"
Jun 03 16:44:48.613625 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:48.613563    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused"
Jun 03 16:44:49.613832 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:49.613738    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused"
Jun 03 16:44:50.613480 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:50.613427    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0"
Jun 03 16:44:50.614638 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:50.614605    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused"
Jun 03 16:44:51.613899 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:51.613848    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused"
Jun 03 16:44:52.613869 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:52.613820    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused"
Jun 03 16:44:53.613630 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:53.613572    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused"
Jun 03 16:44:54.613451 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:54.613400    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused"
Jun 03 16:44:55.613316 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:55.613258    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused"
Jun 03 16:44:56.613149 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:56.613099    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused"
Jun 03 16:44:57.613422 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:57.613381    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused"
Jun 03 16:44:58.746972 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:58.746920    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0"
Jun 03 16:44:58.767326 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:44:58.767241    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 03 16:45:08.754426 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:45:08.754388    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0"
Jun 03 16:45:58.872358 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:45:58.872317    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 16:45:58.872904 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:45:58.872559    2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak" containerID="cri-o://2c28b71376005bd764e2b7f9f2a3a5c802e1fda4d693749baeccfdb7f5c85164" gracePeriod=30
Jun 03 16:46:00.448207 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:00.448169    2569 generic.go:358] "Generic (PLEG): container finished" podID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerID="2c28b71376005bd764e2b7f9f2a3a5c802e1fda4d693749baeccfdb7f5c85164" exitCode=143
Jun 03 16:46:00.448579 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:00.448244    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"eccb3201-6c8a-41ba-8774-5f9c4b6b0638","Type":"ContainerDied","Data":"2c28b71376005bd764e2b7f9f2a3a5c802e1fda4d693749baeccfdb7f5c85164"}
Jun 03 16:46:00.907329 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:00.907304    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 03 16:46:00.923830 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:00.923787    2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhgvj\" (UniqueName: \"kubernetes.io/projected/eccb3201-6c8a-41ba-8774-5f9c4b6b0638-kube-api-access-nhgvj\") pod \"eccb3201-6c8a-41ba-8774-5f9c4b6b0638\" (UID: \"eccb3201-6c8a-41ba-8774-5f9c4b6b0638\") "
Jun 03 16:46:00.926024 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:00.926000    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eccb3201-6c8a-41ba-8774-5f9c4b6b0638-kube-api-access-nhgvj" (OuterVolumeSpecName: "kube-api-access-nhgvj") pod "eccb3201-6c8a-41ba-8774-5f9c4b6b0638" (UID: "eccb3201-6c8a-41ba-8774-5f9c4b6b0638"). InnerVolumeSpecName "kube-api-access-nhgvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 03 16:46:01.024477 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.024438    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhgvj\" (UniqueName: \"kubernetes.io/projected/eccb3201-6c8a-41ba-8774-5f9c4b6b0638-kube-api-access-nhgvj\") on node \"ip-10-0-140-123.ec2.internal\" DevicePath \"\""
Jun 03 16:46:01.452359 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.452283    2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 03 16:46:01.452846 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.452281    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"eccb3201-6c8a-41ba-8774-5f9c4b6b0638","Type":"ContainerDied","Data":"49731b7337ba9be85721b60d3307d1b643f2868d54db0962202b5422b32991eb"}
Jun 03 16:46:01.452846 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.452403    2569 scope.go:117] "RemoveContainer" containerID="2c28b71376005bd764e2b7f9f2a3a5c802e1fda4d693749baeccfdb7f5c85164"
Jun 03 16:46:01.469115 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.469091    2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 16:46:01.472349 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.472327    2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 16:46:01.491267 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.491241    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 16:46:01.491569 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.491552    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak"
Jun 03 16:46:01.491620 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.491570    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak"
Jun 03 16:46:01.491658 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.491648    2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" containerName="keycloak"
Jun 03 16:46:01.493959 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.493937    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 03 16:46:01.496674 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.496652    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\""
Jun 03 16:46:01.496674 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.496664    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\""
Jun 03 16:46:01.496870 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.496692    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-p28sv\""
Jun 03 16:46:01.496870 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.496726    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\""
Jun 03 16:46:01.496870 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.496820    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\""
Jun 03 16:46:01.502650 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.502631    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 16:46:01.529150 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.529127    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/59c862d8-1602-44b2-b31d-a3321f799f16-test-realms\") pod \"maas-keycloak-0\" (UID: \"59c862d8-1602-44b2-b31d-a3321f799f16\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 16:46:01.529284 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.529169    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v4fp\" (UniqueName: \"kubernetes.io/projected/59c862d8-1602-44b2-b31d-a3321f799f16-kube-api-access-8v4fp\") pod \"maas-keycloak-0\" (UID: \"59c862d8-1602-44b2-b31d-a3321f799f16\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 16:46:01.630103 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.630071    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/59c862d8-1602-44b2-b31d-a3321f799f16-test-realms\") pod \"maas-keycloak-0\" (UID: \"59c862d8-1602-44b2-b31d-a3321f799f16\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 16:46:01.630269 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.630121    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8v4fp\" (UniqueName: \"kubernetes.io/projected/59c862d8-1602-44b2-b31d-a3321f799f16-kube-api-access-8v4fp\") pod \"maas-keycloak-0\" (UID: \"59c862d8-1602-44b2-b31d-a3321f799f16\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 16:46:01.630720 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.630700    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/59c862d8-1602-44b2-b31d-a3321f799f16-test-realms\") pod \"maas-keycloak-0\" (UID: \"59c862d8-1602-44b2-b31d-a3321f799f16\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 16:46:01.638143 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.638114    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v4fp\" (UniqueName: \"kubernetes.io/projected/59c862d8-1602-44b2-b31d-a3321f799f16-kube-api-access-8v4fp\") pod \"maas-keycloak-0\" (UID: \"59c862d8-1602-44b2-b31d-a3321f799f16\") " pod="keycloak-system/maas-keycloak-0"
Jun 03 16:46:01.804677 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.804642    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
Jun 03 16:46:01.934436 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.934411    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"]
Jun 03 16:46:01.936217 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:46:01.936188    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c862d8_1602_44b2_b31d_a3321f799f16.slice/crio-552f1c53c97c429210e62363d2aa7485fba164b31b1310edc36a1e16033698c1 WatchSource:0}: Error finding container 552f1c53c97c429210e62363d2aa7485fba164b31b1310edc36a1e16033698c1: Status 404 returned error can't find the container with id 552f1c53c97c429210e62363d2aa7485fba164b31b1310edc36a1e16033698c1
Jun 03 16:46:01.937789 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:01.937775    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 03 16:46:02.459487 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:02.459444    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"59c862d8-1602-44b2-b31d-a3321f799f16","Type":"ContainerStarted","Data":"ee000b18dc2a2bd261f4ec1873392b54f10dc025ae047a5f866e2bb5693f19c0"}
Jun 03 16:46:02.459487 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:02.459488    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"59c862d8-1602-44b2-b31d-a3321f799f16","Type":"ContainerStarted","Data":"552f1c53c97c429210e62363d2aa7485fba164b31b1310edc36a1e16033698c1"}
Jun 03 16:46:02.478670 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:02.478617    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.1721177520000001 podStartE2EDuration="1.478604525s" podCreationTimestamp="2026-06-03 16:46:01 +0000 UTC" firstStartedPulling="2026-06-03 16:46:01.937909682 +0000 UTC m=+731.468429243" lastFinishedPulling="2026-06-03 16:46:02.244396456 +0000 UTC m=+731.774916016" observedRunningTime="2026-06-03 16:46:02.47654452 +0000 UTC m=+732.007064104" watchObservedRunningTime="2026-06-03 16:46:02.478604525 +0000 UTC m=+732.009124107"
Jun 03 16:46:02.805465 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:02.805423    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0"
Jun 03 16:46:02.807034 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:02.806996    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59c862d8-1602-44b2-b31d-a3321f799f16" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.28:9000/health/started\": dial tcp 10.133.0.28:9000: connect: connection refused"
Jun 03 16:46:03.076248 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:03.076157    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eccb3201-6c8a-41ba-8774-5f9c4b6b0638" path="/var/lib/kubelet/pods/eccb3201-6c8a-41ba-8774-5f9c4b6b0638/volumes"
Jun 03 16:46:03.805605 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:03.805553    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59c862d8-1602-44b2-b31d-a3321f799f16" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.28:9000/health/started\": dial tcp 10.133.0.28:9000: connect: connection refused"
Jun 03 16:46:04.805673 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:04.805613    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59c862d8-1602-44b2-b31d-a3321f799f16" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.28:9000/health/started\": dial tcp 10.133.0.28:9000: connect: connection refused"
Jun 03 16:46:05.805501 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:05.805440    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59c862d8-1602-44b2-b31d-a3321f799f16" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.28:9000/health/started\": dial tcp 10.133.0.28:9000: connect: connection refused"
Jun 03 16:46:06.805902 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:06.805840    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59c862d8-1602-44b2-b31d-a3321f799f16" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.28:9000/health/started\": dial tcp 10.133.0.28:9000: connect: connection refused"
Jun 03 16:46:07.806109 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:07.806039    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59c862d8-1602-44b2-b31d-a3321f799f16" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.28:9000/health/started\": dial tcp 10.133.0.28:9000: connect: connection refused"
Jun 03 16:46:08.805645 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:08.805583    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59c862d8-1602-44b2-b31d-a3321f799f16" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.28:9000/health/started\": dial tcp 10.133.0.28:9000: connect: connection refused"
Jun 03 16:46:09.805577 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:09.805511    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59c862d8-1602-44b2-b31d-a3321f799f16" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.28:9000/health/started\": dial tcp 10.133.0.28:9000: connect: connection refused"
Jun 03 16:46:10.805668 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:10.805609    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59c862d8-1602-44b2-b31d-a3321f799f16" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.28:9000/health/started\": dial tcp 10.133.0.28:9000: connect: connection refused"
Jun 03 16:46:11.804941 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:11.804876    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0"
Jun 03 16:46:11.805306 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:11.805272    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59c862d8-1602-44b2-b31d-a3321f799f16" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.28:9000/health/started\": dial tcp 10.133.0.28:9000: connect: connection refused"
Jun 03 16:46:12.805588 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:12.805535    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59c862d8-1602-44b2-b31d-a3321f799f16" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.28:9000/health/started\": dial tcp 10.133.0.28:9000: connect: connection refused"
Jun 03 16:46:13.805749 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:13.805701    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59c862d8-1602-44b2-b31d-a3321f799f16" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.28:9000/health/started\": dial tcp 10.133.0.28:9000: connect: connection refused"
Jun 03 16:46:14.805886 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:14.805833    2569 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59c862d8-1602-44b2-b31d-a3321f799f16" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.28:9000/health/started\": dial tcp 10.133.0.28:9000: connect: connection refused"
Jun 03 16:46:15.929422 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:15.929365    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0"
Jun 03 16:46:15.947637 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:15.947586    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="59c862d8-1602-44b2-b31d-a3321f799f16" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jun 03 16:46:25.935487 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:25.935450    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0"
Jun 03 16:46:45.708439 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.708404    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"]
Jun 03 16:46:45.712360 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.712340    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.716573 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.716546    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jun 03 16:46:45.716573 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.716562    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-djdp4\""
Jun 03 16:46:45.716573 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.716546    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jun 03 16:46:45.716774 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.716586    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jun 03 16:46:45.721316 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.721293    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"]
Jun 03 16:46:45.814809 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.814760    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.814978 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.814874    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.814978 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.814903    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.815054 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.814988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.815054 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.815037    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x5gv\" (UniqueName: \"kubernetes.io/projected/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-kube-api-access-5x5gv\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.815116 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.815064    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.916165 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.916113    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.916165 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.916173    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.916428 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.916195    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.916428 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.916224    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.916428 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.916250    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5x5gv\" (UniqueName: \"kubernetes.io/projected/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-kube-api-access-5x5gv\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.916428 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.916272    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.916648 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.916604    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.916708 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.916646    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.916708 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.916697    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.918532 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.918510    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.918829 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.918790    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:45.924262 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:45.924242    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x5gv\" (UniqueName: \"kubernetes.io/projected/8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5-kube-api-access-5x5gv\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89\" (UID: \"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:46.023761 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:46.023649    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:46:46.156783 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:46.156751    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"]
Jun 03 16:46:46.159107 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:46:46.159074    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a6acf60_cd4c_4d8d_8de2_dd87f4bd83d5.slice/crio-cda8a4deb6ee9e54cad787a6bf20f246b27a878eb801f7d504a8ceb353f40b3d WatchSource:0}: Error finding container cda8a4deb6ee9e54cad787a6bf20f246b27a878eb801f7d504a8ceb353f40b3d: Status 404 returned error can't find the container with id cda8a4deb6ee9e54cad787a6bf20f246b27a878eb801f7d504a8ceb353f40b3d
Jun 03 16:46:46.624215 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:46.624184    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89" event={"ID":"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5","Type":"ContainerStarted","Data":"cda8a4deb6ee9e54cad787a6bf20f246b27a878eb801f7d504a8ceb353f40b3d"}
Jun 03 16:46:50.641441 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:46:50.641403    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89" event={"ID":"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5","Type":"ContainerStarted","Data":"784f1d9f94e9fd08383c5a6b3fb4e103f21e8b9fc6d85e5b8e3948cd2224123d"}
Jun 03 16:47:12.794993 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.794906    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"]
Jun 03 16:47:12.806306 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.806269    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.812537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.811482    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"]
Jun 03 16:47:12.814345 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.814319    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jun 03 16:47:12.856313 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.856281    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e315e40b-a97d-41bb-8cd2-f4da4646840c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.856471 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.856337    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e315e40b-a97d-41bb-8cd2-f4da4646840c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.856471 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.856395    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e315e40b-a97d-41bb-8cd2-f4da4646840c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.856471 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.856458    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75x2k\" (UniqueName: \"kubernetes.io/projected/e315e40b-a97d-41bb-8cd2-f4da4646840c-kube-api-access-75x2k\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.856603 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.856500    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e315e40b-a97d-41bb-8cd2-f4da4646840c-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.856603 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.856547    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e315e40b-a97d-41bb-8cd2-f4da4646840c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.957203 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.957167    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e315e40b-a97d-41bb-8cd2-f4da4646840c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.957203 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.957204    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e315e40b-a97d-41bb-8cd2-f4da4646840c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.957404 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.957225    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-75x2k\" (UniqueName: \"kubernetes.io/projected/e315e40b-a97d-41bb-8cd2-f4da4646840c-kube-api-access-75x2k\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.957404 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.957254    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e315e40b-a97d-41bb-8cd2-f4da4646840c-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.957404 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.957282    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e315e40b-a97d-41bb-8cd2-f4da4646840c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.957404 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.957313    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e315e40b-a97d-41bb-8cd2-f4da4646840c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.957647 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.957626    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e315e40b-a97d-41bb-8cd2-f4da4646840c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.957692 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.957663    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e315e40b-a97d-41bb-8cd2-f4da4646840c-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.957752 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.957735    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e315e40b-a97d-41bb-8cd2-f4da4646840c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.959547 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.959522    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e315e40b-a97d-41bb-8cd2-f4da4646840c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.959783 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.959767    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e315e40b-a97d-41bb-8cd2-f4da4646840c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:12.965829 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:12.965785    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75x2k\" (UniqueName: \"kubernetes.io/projected/e315e40b-a97d-41bb-8cd2-f4da4646840c-kube-api-access-75x2k\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh\" (UID: \"e315e40b-a97d-41bb-8cd2-f4da4646840c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:13.121634 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:13.121591    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:13.250013 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:13.249987    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"]
Jun 03 16:47:13.252095 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:47:13.252068    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode315e40b_a97d_41bb_8cd2_f4da4646840c.slice/crio-4da7bc0d9b1a4e63dbbfbb77ebdcf7886fd59ad54853e4f67fe45f95186eb7fb WatchSource:0}: Error finding container 4da7bc0d9b1a4e63dbbfbb77ebdcf7886fd59ad54853e4f67fe45f95186eb7fb: Status 404 returned error can't find the container with id 4da7bc0d9b1a4e63dbbfbb77ebdcf7886fd59ad54853e4f67fe45f95186eb7fb
Jun 03 16:47:13.716238 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:13.716204    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh" event={"ID":"e315e40b-a97d-41bb-8cd2-f4da4646840c","Type":"ContainerStarted","Data":"a2ac9884db7e4050db2338be8730a5618742cc697d9c258b646ad5c2bd3e27f7"}
Jun 03 16:47:13.716238 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:13.716240    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh" event={"ID":"e315e40b-a97d-41bb-8cd2-f4da4646840c","Type":"ContainerStarted","Data":"4da7bc0d9b1a4e63dbbfbb77ebdcf7886fd59ad54853e4f67fe45f95186eb7fb"}
Jun 03 16:47:16.727024 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:16.726992    2569 generic.go:358] "Generic (PLEG): container finished" podID="e315e40b-a97d-41bb-8cd2-f4da4646840c" containerID="a2ac9884db7e4050db2338be8730a5618742cc697d9c258b646ad5c2bd3e27f7" exitCode=0
Jun 03 16:47:16.727024 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:16.727028    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh" event={"ID":"e315e40b-a97d-41bb-8cd2-f4da4646840c","Type":"ContainerDied","Data":"a2ac9884db7e4050db2338be8730a5618742cc697d9c258b646ad5c2bd3e27f7"}
Jun 03 16:47:21.747861 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:21.747825    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh" event={"ID":"e315e40b-a97d-41bb-8cd2-f4da4646840c","Type":"ContainerStarted","Data":"8913edc40c70b67c57563dd342454f9d807fafd641276a1d63eeadd0835b434c"}
Jun 03 16:47:21.766151 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:21.766099    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh" podStartSLOduration=5.688822276 podStartE2EDuration="9.766085503s" podCreationTimestamp="2026-06-03 16:47:12 +0000 UTC" firstStartedPulling="2026-06-03 16:47:16.727603836 +0000 UTC m=+806.258123396" lastFinishedPulling="2026-06-03 16:47:20.804867059 +0000 UTC m=+810.335386623" observedRunningTime="2026-06-03 16:47:21.765084628 +0000 UTC m=+811.295604211" watchObservedRunningTime="2026-06-03 16:47:21.766085503 +0000 UTC m=+811.296605085"
Jun 03 16:47:23.121884 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:23.121845    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:23.122282 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:23.122144    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:23.135367 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:23.135339    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:23.770814 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:23.770766    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh"
Jun 03 16:47:35.974274 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:35.974231    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"]
Jun 03 16:47:35.976875 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:35.976853    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:35.979670 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:35.979641    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jun 03 16:47:35.985985 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:35.985956    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"]
Jun 03 16:47:36.159236 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.159204    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccb13487-4052-4dc6-b46f-a675e5aadf54-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.159418 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.159249    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccb13487-4052-4dc6-b46f-a675e5aadf54-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.159418 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.159285    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w5r8\" (UniqueName: \"kubernetes.io/projected/ccb13487-4052-4dc6-b46f-a675e5aadf54-kube-api-access-9w5r8\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.159418 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.159316    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccb13487-4052-4dc6-b46f-a675e5aadf54-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.159418 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.159376    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ccb13487-4052-4dc6-b46f-a675e5aadf54-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.159547 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.159459    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccb13487-4052-4dc6-b46f-a675e5aadf54-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.259980 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.259895    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccb13487-4052-4dc6-b46f-a675e5aadf54-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.259980 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.259965    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccb13487-4052-4dc6-b46f-a675e5aadf54-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.260218 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.260004    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccb13487-4052-4dc6-b46f-a675e5aadf54-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.260218 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.260035    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9w5r8\" (UniqueName: \"kubernetes.io/projected/ccb13487-4052-4dc6-b46f-a675e5aadf54-kube-api-access-9w5r8\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.260218 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.260119    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccb13487-4052-4dc6-b46f-a675e5aadf54-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.260418 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.260391    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ccb13487-4052-4dc6-b46f-a675e5aadf54-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.260506 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.260448    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccb13487-4052-4dc6-b46f-a675e5aadf54-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.260696 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.260674    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ccb13487-4052-4dc6-b46f-a675e5aadf54-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.260838 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.260741    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccb13487-4052-4dc6-b46f-a675e5aadf54-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.262428 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.262409    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccb13487-4052-4dc6-b46f-a675e5aadf54-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.262556 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.262539    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccb13487-4052-4dc6-b46f-a675e5aadf54-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.268127 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.268104    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w5r8\" (UniqueName: \"kubernetes.io/projected/ccb13487-4052-4dc6-b46f-a675e5aadf54-kube-api-access-9w5r8\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch\" (UID: \"ccb13487-4052-4dc6-b46f-a675e5aadf54\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.288712 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.288681    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"
Jun 03 16:47:36.421234 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.421209    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch"]
Jun 03 16:47:36.423338 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:47:36.423315    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccb13487_4052_4dc6_b46f_a675e5aadf54.slice/crio-dbb73ffc3308ca000d8c8fce0dbc250582b53e48f26ef15def4cc69727f0c055 WatchSource:0}: Error finding container dbb73ffc3308ca000d8c8fce0dbc250582b53e48f26ef15def4cc69727f0c055: Status 404 returned error can't find the container with id dbb73ffc3308ca000d8c8fce0dbc250582b53e48f26ef15def4cc69727f0c055
Jun 03 16:47:36.801349 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.801308    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch" event={"ID":"ccb13487-4052-4dc6-b46f-a675e5aadf54","Type":"ContainerStarted","Data":"fc3ff75d9a1a122af36f68a8a51e1c6f10b72e9561567dcf33aa174a1e3d5eb5"}
Jun 03 16:47:36.801522 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:36.801354    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch" event={"ID":"ccb13487-4052-4dc6-b46f-a675e5aadf54","Type":"ContainerStarted","Data":"dbb73ffc3308ca000d8c8fce0dbc250582b53e48f26ef15def4cc69727f0c055"}
Jun 03 16:47:53.860974 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:53.860941    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89_8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5/storage-initializer/0.log"
Jun 03 16:47:53.861385 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:53.860993    2569 generic.go:358] "Generic (PLEG): container finished" podID="8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5" containerID="784f1d9f94e9fd08383c5a6b3fb4e103f21e8b9fc6d85e5b8e3948cd2224123d" exitCode=1
Jun 03 16:47:53.861385 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:53.861067    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89" event={"ID":"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5","Type":"ContainerDied","Data":"784f1d9f94e9fd08383c5a6b3fb4e103f21e8b9fc6d85e5b8e3948cd2224123d"}
Jun 03 16:47:54.866537 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:54.866509    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89_8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5/storage-initializer/0.log"
Jun 03 16:47:54.866976 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:54.866574    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89" event={"ID":"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5","Type":"ContainerStarted","Data":"a9fa5e9f42ea18978fb6a23598a1f82883f718dc0e8861c0f30b0b5199b39797"}
Jun 03 16:47:59.886329 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:59.886300    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89_8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5/storage-initializer/0.log"
Jun 03 16:47:59.886832 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:59.886341    2569 generic.go:358] "Generic (PLEG): container finished" podID="8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5" containerID="a9fa5e9f42ea18978fb6a23598a1f82883f718dc0e8861c0f30b0b5199b39797" exitCode=0
Jun 03 16:47:59.886832 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:59.886429    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89" event={"ID":"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5","Type":"ContainerDied","Data":"a9fa5e9f42ea18978fb6a23598a1f82883f718dc0e8861c0f30b0b5199b39797"}
Jun 03 16:47:59.886832 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:59.886488    2569 scope.go:117] "RemoveContainer" containerID="784f1d9f94e9fd08383c5a6b3fb4e103f21e8b9fc6d85e5b8e3948cd2224123d"
Jun 03 16:47:59.886953 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:47:59.886940    2569 scope.go:117] "RemoveContainer" containerID="784f1d9f94e9fd08383c5a6b3fb4e103f21e8b9fc6d85e5b8e3948cd2224123d"
Jun 03 16:47:59.896904 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:47:59.896869    2569 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89_llm_8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5_0 in pod sandbox cda8a4deb6ee9e54cad787a6bf20f246b27a878eb801f7d504a8ceb353f40b3d from index: no such id: '784f1d9f94e9fd08383c5a6b3fb4e103f21e8b9fc6d85e5b8e3948cd2224123d'" containerID="784f1d9f94e9fd08383c5a6b3fb4e103f21e8b9fc6d85e5b8e3948cd2224123d"
Jun 03 16:47:59.896976 ip-10-0-140-123 kubenswrapper[2569]: E0603 16:47:59.896928    2569 kuberuntime_container.go:951] "Unhandled Error" err="failed to remove pod init container \"storage-initializer\": rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89_llm_8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5_0 in pod sandbox cda8a4deb6ee9e54cad787a6bf20f246b27a878eb801f7d504a8ceb353f40b3d from index: no such id: '784f1d9f94e9fd08383c5a6b3fb4e103f21e8b9fc6d85e5b8e3948cd2224123d'; Skipping pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89_llm(8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5)\"" logger="UnhandledError"
Jun 03 16:48:09.927392 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:48:09.927353    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89" event={"ID":"8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5","Type":"ContainerStarted","Data":"2d36ec06a8c0e1c0c3d8d7c89bdf49aa6f902314005d48c762abab5e82f30948"}
Jun 03 16:48:09.947038 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:48:09.946986    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89" podStartSLOduration=1.708029132 podStartE2EDuration="1m24.946962182s" podCreationTimestamp="2026-06-03 16:46:45 +0000 UTC" firstStartedPulling="2026-06-03 16:46:46.160991144 +0000 UTC m=+775.691510705" lastFinishedPulling="2026-06-03 16:48:09.399924179 +0000 UTC m=+858.930443755" observedRunningTime="2026-06-03 16:48:09.946247836 +0000 UTC m=+859.476767421" watchObservedRunningTime="2026-06-03 16:48:09.946962182 +0000 UTC m=+859.477481773"
Jun 03 16:48:16.024253 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:48:16.024211    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:48:16.024732 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:48:16.024291    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:48:16.039402 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:48:16.039383    2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:48:16.961840 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:48:16.961813    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89"
Jun 03 16:54:02.761287 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:02.761256    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch_ccb13487-4052-4dc6-b46f-a675e5aadf54/storage-initializer/0.log"
Jun 03 16:54:08.264226 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:08.264199    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-99f686c75-zzx5m_93668359-87fa-4669-b741-9f66c4769b5d/nginx/0.log"
Jun 03 16:54:08.374421 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:08.374397    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-7c99ffcdcd-pswtv_ad7675de-c2c2-42c6-bb89-19fc317cd696/manager/0.log"
Jun 03 16:54:08.494403 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:08.494355    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-c6b9847d7-n7mdv_49a57957-b12d-4a75-979b-814b7f656492/manager/0.log"
Jun 03 16:54:08.855202 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:08.855177    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-6577596cd5-kqzpv_e25b10b0-b604-43a8-9d61-9ef957201a7e/server/0.log"
Jun 03 16:54:08.967864 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:08.967827    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-5f8mp_c2f6ac50-09f1-4f61-8160-530a2102718b/manager/0.log"
Jun 03 16:54:09.321866 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:09.321841    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f84758474-skksw_4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a/copy-manifests/0.log"
Jun 03 16:54:09.335074 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:09.335048    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f84758474-skksw_4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a/manager/0.log"
Jun 03 16:54:11.271267 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:11.271231    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-49f8x_90a7e488-0ee4-486d-b84a-b2ca922314f7/manager/0.log"
Jun 03 16:54:11.495397 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:11.495362    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-mrqnn_1ccc8c72-c970-40cc-8665-96a0a2195779/manager/0.log"
Jun 03 16:54:11.955137 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:11.955107    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-rs8sk_6465d0cd-83f6-4b71-b5fa-43e4017ab0e5/discovery/0.log"
Jun 03 16:54:12.173722 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:12.173679    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-598f4f7759-pv7cn_6710ee7b-cbd5-4791-95c0-2c31b54e66b2/kube-auth-proxy/0.log"
Jun 03 16:54:12.867604 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:12.867576    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89_8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5/storage-initializer/1.log"
Jun 03 16:54:12.875000 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:12.874969    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-5fv89_8a6acf60-cd4c-4d8d-8de2-dd87f4bd83d5/main/0.log"
Jun 03 16:54:13.214938 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:13.214845    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh_e315e40b-a97d-41bb-8cd2-f4da4646840c/storage-initializer/0.log"
Jun 03 16:54:13.221762 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:13.221740    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9k75mh_e315e40b-a97d-41bb-8cd2-f4da4646840c/main/0.log"
Jun 03 16:54:13.455986 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:13.455960    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-d8bdbcb5c-dkwch_ccb13487-4052-4dc6-b46f-a675e5aadf54/storage-initializer/0.log"
Jun 03 16:54:20.358561 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:20.358509    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cw9c7_700242dd-718f-4d0f-9a09-410717e2a0fa/global-pull-secret-syncer/0.log"
Jun 03 16:54:20.441636 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:20.441592    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-j5pvb_5023b960-cc21-4146-bab5-a63abb187a37/konnectivity-agent/0.log"
Jun 03 16:54:20.531849 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:20.531818    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-123.ec2.internal_9ef217c3fb12636ad0c2fc98457b1b27/haproxy/0.log"
Jun 03 16:54:24.891687 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:24.891657    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-49f8x_90a7e488-0ee4-486d-b84a-b2ca922314f7/manager/0.log"
Jun 03 16:54:24.942829 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:24.942784    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-mrqnn_1ccc8c72-c970-40cc-8665-96a0a2195779/manager/0.log"
Jun 03 16:54:26.643420 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:26.643391    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n5ql5_44f9713b-30e4-4227-8250-ae656f6534bd/node-exporter/0.log"
Jun 03 16:54:26.672764 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:26.672739    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n5ql5_44f9713b-30e4-4227-8250-ae656f6534bd/kube-rbac-proxy/0.log"
Jun 03 16:54:26.719115 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:26.719087    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n5ql5_44f9713b-30e4-4227-8250-ae656f6534bd/init-textfile/0.log"
Jun 03 16:54:29.973427 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:29.973349    2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"]
Jun 03 16:54:29.978091 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:29.978069    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:29.980838 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:29.980779    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s47xq\"/\"openshift-service-ca.crt\""
Jun 03 16:54:29.981960 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:29.981941    2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s47xq\"/\"default-dockercfg-dsvp8\""
Jun 03 16:54:29.982071 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:29.981989    2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s47xq\"/\"kube-root-ca.crt\""
Jun 03 16:54:29.984184 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:29.984164    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"]
Jun 03 16:54:30.030161 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.030118    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545-podres\") pod \"perf-node-gather-daemonset-dpgtg\" (UID: \"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545\") " pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:30.030351 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.030177    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545-sys\") pod \"perf-node-gather-daemonset-dpgtg\" (UID: \"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545\") " pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:30.030351 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.030265    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtk2t\" (UniqueName: \"kubernetes.io/projected/e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545-kube-api-access-qtk2t\") pod \"perf-node-gather-daemonset-dpgtg\" (UID: \"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545\") " pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:30.030351 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.030298    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545-proc\") pod \"perf-node-gather-daemonset-dpgtg\" (UID: \"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545\") " pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:30.030351 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.030320    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545-lib-modules\") pod \"perf-node-gather-daemonset-dpgtg\" (UID: \"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545\") " pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:30.131614 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.131581    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qtk2t\" (UniqueName: \"kubernetes.io/projected/e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545-kube-api-access-qtk2t\") pod \"perf-node-gather-daemonset-dpgtg\" (UID: \"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545\") " pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:30.131854 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.131623    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545-proc\") pod \"perf-node-gather-daemonset-dpgtg\" (UID: \"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545\") " pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:30.131854 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.131656    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545-lib-modules\") pod \"perf-node-gather-daemonset-dpgtg\" (UID: \"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545\") " pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:30.131854 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.131698    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545-podres\") pod \"perf-node-gather-daemonset-dpgtg\" (UID: \"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545\") " pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:30.131854 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.131734    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545-proc\") pod \"perf-node-gather-daemonset-dpgtg\" (UID: \"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545\") " pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:30.131854 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.131743    2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545-sys\") pod \"perf-node-gather-daemonset-dpgtg\" (UID: \"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545\") " pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:30.131854 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.131791    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545-sys\") pod \"perf-node-gather-daemonset-dpgtg\" (UID: \"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545\") " pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:30.132092 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.131888    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545-lib-modules\") pod \"perf-node-gather-daemonset-dpgtg\" (UID: \"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545\") " pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:30.132092 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.131887    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545-podres\") pod \"perf-node-gather-daemonset-dpgtg\" (UID: \"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545\") " pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:30.139780 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.139760    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtk2t\" (UniqueName: \"kubernetes.io/projected/e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545-kube-api-access-qtk2t\") pod \"perf-node-gather-daemonset-dpgtg\" (UID: \"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545\") " pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:30.289297 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.289204    2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:30.418043 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.418018    2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"]
Jun 03 16:54:30.420621 ip-10-0-140-123 kubenswrapper[2569]: W0603 16:54:30.420584    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode743e0ec_2ca6_4fe8_a202_8d4eeb6f5545.slice/crio-2490bfba15aae0bb003b7a9d9145ce2cd49ae79d0197dad836b2818fff5b53fe WatchSource:0}: Error finding container 2490bfba15aae0bb003b7a9d9145ce2cd49ae79d0197dad836b2818fff5b53fe: Status 404 returned error can't find the container with id 2490bfba15aae0bb003b7a9d9145ce2cd49ae79d0197dad836b2818fff5b53fe
Jun 03 16:54:30.422284 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:30.422268    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 03 16:54:31.081841 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:31.081810    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x7fvf_9eb09939-3056-4ca9-8d22-64cd87cf4f8e/dns/0.log"
Jun 03 16:54:31.108087 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:31.108064    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x7fvf_9eb09939-3056-4ca9-8d22-64cd87cf4f8e/kube-rbac-proxy/0.log"
Jun 03 16:54:31.133516 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:31.133491    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vzp5g_fefcdfd2-e498-45e1-ae6d-fa8431ba97fa/dns-node-resolver/0.log"
Jun 03 16:54:31.199747 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:31.199713    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg" event={"ID":"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545","Type":"ContainerStarted","Data":"310f5cd2fd10e67471e55cfa5ceb6bf9bf5d2adc938c8b82bf8216725ca4d2f8"}
Jun 03 16:54:31.199747 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:31.199749    2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg" event={"ID":"e743e0ec-2ca6-4fe8-a202-8d4eeb6f5545","Type":"ContainerStarted","Data":"2490bfba15aae0bb003b7a9d9145ce2cd49ae79d0197dad836b2818fff5b53fe"}
Jun 03 16:54:31.199952 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:31.199773    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:31.215292 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:31.215250    2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg" podStartSLOduration=2.21523626 podStartE2EDuration="2.21523626s" podCreationTimestamp="2026-06-03 16:54:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:54:31.214268034 +0000 UTC m=+1240.744787620" watchObservedRunningTime="2026-06-03 16:54:31.21523626 +0000 UTC m=+1240.745755843"
Jun 03 16:54:31.659392 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:31.659364    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ccbzr_c0fd9ccf-e728-4aba-9249-daed366df1e5/node-ca/0.log"
Jun 03 16:54:32.602691 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:32.602666    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-rs8sk_6465d0cd-83f6-4b71-b5fa-43e4017ab0e5/discovery/0.log"
Jun 03 16:54:32.652729 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:32.652703    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-598f4f7759-pv7cn_6710ee7b-cbd5-4791-95c0-2c31b54e66b2/kube-auth-proxy/0.log"
Jun 03 16:54:33.251538 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:33.251511    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tdqmj_bd8eaa6c-3029-4458-9d0c-13933d35c3c6/serve-healthcheck-canary/0.log"
Jun 03 16:54:33.869947 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:33.869918    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zx2cx_636a1bf9-f7d4-4107-b669-30d08086d657/kube-rbac-proxy/0.log"
Jun 03 16:54:33.892010 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:33.891984    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zx2cx_636a1bf9-f7d4-4107-b669-30d08086d657/exporter/0.log"
Jun 03 16:54:33.916838 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:33.916812    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zx2cx_636a1bf9-f7d4-4107-b669-30d08086d657/extractor/0.log"
Jun 03 16:54:35.821904 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:35.821872    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-99f686c75-zzx5m_93668359-87fa-4669-b741-9f66c4769b5d/nginx/0.log"
Jun 03 16:54:35.846234 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:35.846212    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-7c99ffcdcd-pswtv_ad7675de-c2c2-42c6-bb89-19fc317cd696/manager/0.log"
Jun 03 16:54:35.880207 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:35.880176    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-c6b9847d7-n7mdv_49a57957-b12d-4a75-979b-814b7f656492/manager/0.log"
Jun 03 16:54:35.945323 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:35.945281    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-6577596cd5-kqzpv_e25b10b0-b604-43a8-9d61-9ef957201a7e/server/0.log"
Jun 03 16:54:35.992693 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:35.992661    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-5f8mp_c2f6ac50-09f1-4f61-8160-530a2102718b/manager/0.log"
Jun 03 16:54:36.165160 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:36.165079    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f84758474-skksw_4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a/manager/0.log"
Jun 03 16:54:36.183011 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:36.182989    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f84758474-skksw_4a4bbbec-0955-4a88-b1ed-ed5a6b1f011a/copy-manifests/0.log"
Jun 03 16:54:37.212020 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:37.211979    2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s47xq/perf-node-gather-daemonset-dpgtg"
Jun 03 16:54:37.527562 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:37.527484    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-gdcqw_31b191d4-a9b4-40a8-bca3-a9b538f648a8/openshift-lws-operator/0.log"
Jun 03 16:54:43.948619 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:43.948588    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qmmz5_f8b6b06a-4893-46c5-b030-88a2c01abe88/kube-multus-additional-cni-plugins/0.log"
Jun 03 16:54:43.971246 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:43.971220    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qmmz5_f8b6b06a-4893-46c5-b030-88a2c01abe88/egress-router-binary-copy/0.log"
Jun 03 16:54:43.992234 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:43.992213    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qmmz5_f8b6b06a-4893-46c5-b030-88a2c01abe88/cni-plugins/0.log"
Jun 03 16:54:44.013249 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:44.013227    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qmmz5_f8b6b06a-4893-46c5-b030-88a2c01abe88/bond-cni-plugin/0.log"
Jun 03 16:54:44.036551 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:44.036521    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qmmz5_f8b6b06a-4893-46c5-b030-88a2c01abe88/routeoverride-cni/0.log"
Jun 03 16:54:44.060074 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:44.060050    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qmmz5_f8b6b06a-4893-46c5-b030-88a2c01abe88/whereabouts-cni-bincopy/0.log"
Jun 03 16:54:44.083657 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:44.083634    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qmmz5_f8b6b06a-4893-46c5-b030-88a2c01abe88/whereabouts-cni/0.log"
Jun 03 16:54:44.146381 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:44.146346    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-s2z68_a5d651cc-b55b-4806-96ef-1e93043eb8e0/kube-multus/0.log"
Jun 03 16:54:44.286533 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:44.286459    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-q8x4r_d1292e61-8cac-4672-af45-d434487b3426/network-metrics-daemon/0.log"
Jun 03 16:54:44.306455 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:44.306432    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-q8x4r_d1292e61-8cac-4672-af45-d434487b3426/kube-rbac-proxy/0.log"
Jun 03 16:54:45.622126 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:45.622095    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xh2c4_bbeda011-ea40-4374-b7bf-cb93cae95d56/ovn-controller/0.log"
Jun 03 16:54:45.648639 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:45.648609    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xh2c4_bbeda011-ea40-4374-b7bf-cb93cae95d56/ovn-acl-logging/0.log"
Jun 03 16:54:45.668167 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:45.668144    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xh2c4_bbeda011-ea40-4374-b7bf-cb93cae95d56/kube-rbac-proxy-node/0.log"
Jun 03 16:54:45.691168 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:45.691143    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xh2c4_bbeda011-ea40-4374-b7bf-cb93cae95d56/kube-rbac-proxy-ovn-metrics/0.log"
Jun 03 16:54:45.725890 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:45.725863    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xh2c4_bbeda011-ea40-4374-b7bf-cb93cae95d56/northd/0.log"
Jun 03 16:54:45.763511 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:45.763481    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xh2c4_bbeda011-ea40-4374-b7bf-cb93cae95d56/nbdb/0.log"
Jun 03 16:54:45.803594 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:45.803564    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xh2c4_bbeda011-ea40-4374-b7bf-cb93cae95d56/sbdb/0.log"
Jun 03 16:54:45.916523 ip-10-0-140-123 kubenswrapper[2569]: I0603 16:54:45.916435    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xh2c4_bbeda011-ea40-4374-b7bf-cb93cae95d56/ovnkube-controller/0.log"