May 25 16:34:13.986833 ip-10-0-135-137 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
May 25 16:34:13.986846 ip-10-0-135-137 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
May 25 16:34:13.986855 ip-10-0-135-137 systemd[1]: kubelet.service: Failed with result 'resources'.
May 25 16:34:13.987179 ip-10-0-135-137 systemd[1]: Failed to start Kubernetes Kubelet.
May 25 16:34:24.113035 ip-10-0-135-137 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
May 25 16:34:24.113055 ip-10-0-135-137 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 504840cd5fb94595bff2425a9d1dc292 --
May 25 16:36:34.520191 ip-10-0-135-137 systemd[1]: Starting Kubernetes Kubelet...
May 25 16:36:35.026605 ip-10-0-135-137 kubenswrapper[2554]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 25 16:36:35.026605 ip-10-0-135-137 kubenswrapper[2554]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 25 16:36:35.026605 ip-10-0-135-137 kubenswrapper[2554]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 25 16:36:35.026605 ip-10-0-135-137 kubenswrapper[2554]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 25 16:36:35.026605 ip-10-0-135-137 kubenswrapper[2554]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 25 16:36:35.027806 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.027459    2554 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 25 16:36:35.030679 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030663    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 16:36:35.030679 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030679    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030683    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030687    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030690    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030707    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030712    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030717    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030720    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030723    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030726    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030729    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030734    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030738    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030741    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030744    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030747    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030749    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030752    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030755    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 16:36:35.030750 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030758    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030761    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030764    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030767    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030769    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030772    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030774    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030777    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030779    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030781    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030785    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030788    2554 feature_gate.go:328] unrecognized feature gate: Example2
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030790    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030793    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030795    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030798    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030800    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030803    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030806    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030808    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 16:36:35.031214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030811    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030813    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030816    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030818    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030820    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030823    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030826    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030828    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030831    2554 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030833    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030836    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030838    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030841    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030843    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030847    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030849    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030852    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030855    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030858    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030860    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 16:36:35.031856 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030875    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030878    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030880    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030883    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030885    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030888    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030891    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030893    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030897    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030900    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030902    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030905    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030908    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030911    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030913    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030916    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030919    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030921    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030923    2554 feature_gate.go:328] unrecognized feature gate: Example
May 25 16:36:35.032540 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030926    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 16:36:35.033017 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030928    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 16:36:35.033017 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030931    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 16:36:35.033017 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030933    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 16:36:35.033017 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030936    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 16:36:35.033017 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030938    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 16:36:35.033017 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.030940    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 16:36:35.034448 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034429    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 16:36:35.034448 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034446    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 16:36:35.034448 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034450    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 16:36:35.034448 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034453    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034457    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034460    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034463    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034466    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034469    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034471    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034474    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034477    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034479    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034482    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034485    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034488    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034490    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034492    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034495    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034498    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034500    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034503    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034506    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 16:36:35.034597 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034509    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034511    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034514    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034518    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034521    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034524    2554 feature_gate.go:328] unrecognized feature gate: Example
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034526    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034530    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034533    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034535    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034538    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034542    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034547    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034550    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034553    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034556    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034559    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034561    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034564    2554 feature_gate.go:328] unrecognized feature gate: Example2
May 25 16:36:35.035123 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034567    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034569    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034572    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034574    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034577    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034579    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034582    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034584    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034588    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034590    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034593    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034596    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034599    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034602    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034604    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034607    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034610    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034613    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034615    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034618    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 16:36:35.035604 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034621    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034624    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034627    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034629    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034632    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034635    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034638    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034641    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034644    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034647    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034649    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034652    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034654    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034657    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034659    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034662    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034665    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034668    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034670    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034674    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 16:36:35.036197 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034678    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034681    2554 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034683    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.034686    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034762    2554 flags.go:64] FLAG: --address="0.0.0.0"
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034770    2554 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034778    2554 flags.go:64] FLAG: --anonymous-auth="true"
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034783    2554 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034788    2554 flags.go:64] FLAG: --authentication-token-webhook="false"
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034791    2554 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034795    2554 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034800    2554 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034804    2554 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034807    2554 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034811    2554 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034814    2554 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034817    2554 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034820    2554 flags.go:64] FLAG: --cgroup-root=""
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034823    2554 flags.go:64] FLAG: --cgroups-per-qos="true"
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034826    2554 flags.go:64] FLAG: --client-ca-file=""
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034829    2554 flags.go:64] FLAG: --cloud-config=""
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034832    2554 flags.go:64] FLAG: --cloud-provider="external"
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034835    2554 flags.go:64] FLAG: --cluster-dns="[]"
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034839    2554 flags.go:64] FLAG: --cluster-domain=""
May 25 16:36:35.036697 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034842    2554 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034845    2554 flags.go:64] FLAG: --config-dir=""
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034848    2554 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034851    2554 flags.go:64] FLAG: --container-log-max-files="5"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034855    2554 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034859    2554 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034862    2554 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034879    2554 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034882    2554 flags.go:64] FLAG: --contention-profiling="false"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034885    2554 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034887    2554 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034891    2554 flags.go:64] FLAG: --cpu-manager-policy="none"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034893    2554 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034898    2554 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034901    2554 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034904    2554 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034906    2554 flags.go:64] FLAG: --enable-load-reader="false"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034910    2554 flags.go:64] FLAG: --enable-server="true"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034912    2554 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034918    2554 flags.go:64] FLAG: --event-burst="100"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034921    2554 flags.go:64] FLAG: --event-qps="50"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034924    2554 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034927    2554 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034930    2554 flags.go:64] FLAG: --eviction-hard=""
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034934    2554 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 25 16:36:35.037339 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034937    2554 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034940    2554 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034944    2554 flags.go:64] FLAG: --eviction-soft=""
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034947    2554 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034950    2554 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034953    2554 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034956    2554 flags.go:64] FLAG: --experimental-mounter-path=""
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034958    2554 flags.go:64] FLAG: --fail-cgroupv1="false"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034961    2554 flags.go:64] FLAG: --fail-swap-on="true"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034964    2554 flags.go:64] FLAG: --feature-gates=""
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034968    2554 flags.go:64] FLAG: --file-check-frequency="20s"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034971    2554 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034974    2554 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034978    2554 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034981    2554 flags.go:64] FLAG: --healthz-port="10248"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034984    2554 flags.go:64] FLAG: --help="false"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034987    2554 flags.go:64] FLAG: --hostname-override="ip-10-0-135-137.ec2.internal"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034990    2554 flags.go:64] FLAG: --housekeeping-interval="10s"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034993    2554 flags.go:64] FLAG: --http-check-frequency="20s"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034996    2554 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.034999    2554 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035002    2554 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035006    2554 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 25 16:36:35.037975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035008    2554 flags.go:64] FLAG: --image-service-endpoint=""
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035011    2554 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035014    2554 flags.go:64] FLAG: --kube-api-burst="100"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035018    2554 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035021    2554 flags.go:64] FLAG: --kube-api-qps="50"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035024    2554 flags.go:64] FLAG: --kube-reserved=""
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035027    2554 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035030    2554 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035033    2554 flags.go:64] FLAG: --kubelet-cgroups=""
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035036    2554 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035039    2554 flags.go:64] FLAG: --lock-file=""
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035042    2554 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035045    2554 flags.go:64] FLAG: --log-flush-frequency="5s"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035048    2554 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035054    2554 flags.go:64] FLAG: --log-json-split-stream="false"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035057    2554 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035060    2554 flags.go:64] FLAG: --log-text-split-stream="false"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035063    2554 flags.go:64] FLAG: --logging-format="text"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035066    2554 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035069    2554 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035072    2554 flags.go:64] FLAG: --manifest-url=""
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035075    2554 flags.go:64] FLAG: --manifest-url-header=""
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035080    2554 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035083    2554 flags.go:64] FLAG: --max-open-files="1000000"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035088    2554 flags.go:64] FLAG: --max-pods="110"
May 25 16:36:35.038530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035091    2554 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035094    2554 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035097    2554 flags.go:64] FLAG: --memory-manager-policy="None"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035100    2554 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035103    2554 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035106    2554 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035110    2554 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035118    2554 flags.go:64] FLAG: --node-status-max-images="50"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035121    2554 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035125    2554 flags.go:64] FLAG: --oom-score-adj="-999"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035128    2554 flags.go:64] FLAG: --pod-cidr=""
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035131    2554 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035137    2554 flags.go:64] FLAG: --pod-manifest-path=""
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035140    2554 flags.go:64] FLAG: --pod-max-pids="-1"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035144    2554 flags.go:64] FLAG: --pods-per-core="0"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035147    2554 flags.go:64] FLAG: --port="10250"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035150    2554 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035153    2554 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-042b4278ba94ff1c0"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035156    2554 flags.go:64] FLAG: --qos-reserved=""
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035164    2554 flags.go:64] FLAG: --read-only-port="10255"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035167    2554 flags.go:64] FLAG: --register-node="true"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035170    2554 flags.go:64] FLAG: --register-schedulable="true"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035173    2554 flags.go:64] FLAG: --register-with-taints=""
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035177    2554 flags.go:64] FLAG: --registry-burst="10"
May 25 16:36:35.039173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035180    2554 flags.go:64] FLAG: --registry-qps="5"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035183    2554 flags.go:64] FLAG: --reserved-cpus=""
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035186    2554 flags.go:64] FLAG: --reserved-memory=""
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035189    2554 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035193    2554 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035196    2554 flags.go:64] FLAG: --rotate-certificates="false"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035198    2554 flags.go:64] FLAG: --rotate-server-certificates="false"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035202    2554 flags.go:64] FLAG: --runonce="false"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035205    2554 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035208    2554 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035211    2554 flags.go:64] FLAG: --seccomp-default="false"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035214    2554 flags.go:64] FLAG: --serialize-image-pulls="true"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035217    2554 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035220    2554 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035224    2554 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035227    2554 flags.go:64] FLAG: --storage-driver-password="root"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035229    2554 flags.go:64] FLAG: --storage-driver-secure="false"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035233    2554 flags.go:64] FLAG: --storage-driver-table="stats"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035235    2554 flags.go:64] FLAG: --storage-driver-user="root"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035238    2554 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035241    2554 flags.go:64] FLAG: --sync-frequency="1m0s"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035244    2554 flags.go:64] FLAG: --system-cgroups=""
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035247    2554 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035252    2554 flags.go:64] FLAG: --system-reserved-cgroup=""
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035255    2554 flags.go:64] FLAG: --tls-cert-file=""
May 25 16:36:35.039746 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035258    2554 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035261    2554 flags.go:64] FLAG: --tls-min-version=""
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035265    2554 flags.go:64] FLAG: --tls-private-key-file=""
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035269    2554 flags.go:64] FLAG: --topology-manager-policy="none"
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035272    2554 flags.go:64] FLAG: --topology-manager-policy-options=""
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035275    2554 flags.go:64] FLAG: --topology-manager-scope="container"
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035277    2554 flags.go:64] FLAG: --v="2"
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035281    2554 flags.go:64] FLAG: --version="false"
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035286    2554 flags.go:64] FLAG: --vmodule=""
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035291    2554 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.035294    2554 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035386    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035389    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035392    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035396    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035399    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035402    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035406    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035409    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035412    2554 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035415    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035418    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 16:36:35.040403 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035421    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035424    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035427    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035430    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035433    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035436    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035439    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035442    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035446    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035450    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035452    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035455    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035461    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035464    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035467    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035470    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035473    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035476    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035478    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 16:36:35.040962 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035481    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035484    2554 feature_gate.go:328] unrecognized feature gate: Example
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035487    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035489    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035492    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035495    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035498    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035501    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035503    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035506    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035509    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035512    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035514    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035517    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035519    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035522    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035524    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035527    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035530    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035532    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 16:36:35.041462 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035535    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035538    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035540    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035543    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035545    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035549    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035552    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035554    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035557    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035559    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035562    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035565    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035569    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035572    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035575    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035578    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035581    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035584    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035586    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 16:36:35.042016 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035589    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 16:36:35.042481 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035592    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 16:36:35.042481 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035595    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 16:36:35.042481 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035597    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 16:36:35.042481 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035600    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 16:36:35.042481 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035602    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 16:36:35.042481 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035605    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 16:36:35.042481 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035608    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 16:36:35.042481 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035610    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 16:36:35.042481 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035613    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 16:36:35.042481 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035616    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 16:36:35.042481 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035618    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 16:36:35.042481 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035621    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 16:36:35.042481 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035623    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 16:36:35.042481 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035626    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 16:36:35.042481 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035628    2554 feature_gate.go:328] unrecognized feature gate: Example2
May 25 16:36:35.042481 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.035631    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 16:36:35.042889 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.036430    2554 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 25 16:36:35.043351 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.043328    2554 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 25 16:36:35.043388 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.043352    2554 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 25 16:36:35.043417 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043403    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 16:36:35.043417 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043409    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 16:36:35.043417 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043413    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 16:36:35.043417 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043416    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 16:36:35.043417 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043419    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043423    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043426    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043429    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043432    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043435    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043438    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043441    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043445    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043447    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043450    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043453    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043455    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043458    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043461    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043464    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043466    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043469    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043472    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 16:36:35.043549 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043474    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043477    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043479    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043482    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043486    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043491    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043493    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043496    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043500    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043502    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043505    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043508    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043510    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043513    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043515    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043519    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043522    2554 feature_gate.go:328] unrecognized feature gate: Example
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043524    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043527    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 16:36:35.044058 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043530    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043533    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043536    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043538    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043541    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043544    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043547    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043549    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043552    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043555    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043557    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043560    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043564    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043567    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043569    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043572    2554 feature_gate.go:328] unrecognized feature gate: Example2
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043574    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043577    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043580    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043583    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 16:36:35.044520 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043585    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043588    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043591    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043594    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043596    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043599    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043601    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043604    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043607    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043609    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043612    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043614    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043617    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043619    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043621    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043624    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043626    2554 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043629    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043633    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 16:36:35.045037 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043636    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 16:36:35.045506 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043639    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 16:36:35.045506 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043642    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 16:36:35.045506 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043645    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 16:36:35.045506 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043647    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 16:36:35.045506 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.043653    2554 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 25 16:36:35.045506 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043753    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 16:36:35.045506 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043757    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 16:36:35.045506 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043761    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 16:36:35.045506 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043766    2554 feature_gate.go:328] unrecognized feature gate: Example
May 25 16:36:35.045506 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043769    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 16:36:35.045506 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043772    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 16:36:35.045506 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043775    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 16:36:35.045506 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043777    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 16:36:35.045506 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043780    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 16:36:35.045506 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043783    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043787    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043790    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043792    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043795    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043797    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043801    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043803    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043806    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043809    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043811    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043814    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043817    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043819    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043822    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043825    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043827    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043830    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043832    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043835    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 16:36:35.045885 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043837    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043840    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043842    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043845    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043848    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043851    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043853    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043856    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043858    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043861    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043878    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043882    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043885    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043887    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043890    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043893    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043896    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043898    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043901    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043903    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 16:36:35.046386 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043906    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043909    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043911    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043914    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043916    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043919    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043922    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043924    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043927    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043929    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043931    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043934    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043936    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043939    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043941    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043944    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043947    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043949    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043953    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 16:36:35.046892 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043956    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043958    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043961    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043964    2554 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043966    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043969    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043972    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043974    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043977    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043980    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043982    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043985    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043988    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043991    2554 feature_gate.go:328] unrecognized feature gate: Example2
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043993    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043995    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.043998    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 16:36:35.047353 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:35.044000    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 16:36:35.047757 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.044005    2554 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 25 16:36:35.047757 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.044163    2554 server.go:962] "Client rotation is on, will bootstrap in background"
May 25 16:36:35.047817 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.047809    2554 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 25 16:36:35.048970 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.048956    2554 server.go:1019] "Starting client certificate rotation"
May 25 16:36:35.049075 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.049057    2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 25 16:36:35.049105 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.049095    2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 25 16:36:35.077072 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.077042    2554 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 25 16:36:35.079065 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.079047    2554 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 25 16:36:35.096409 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.096383    2554 log.go:25] "Validated CRI v1 runtime API"
May 25 16:36:35.106170 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.106149    2554 log.go:25] "Validated CRI v1 image API"
May 25 16:36:35.107757 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.107737    2554 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 25 16:36:35.112178 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.112155    2554 fs.go:135] Filesystem UUIDs: map[4c87f902-c79d-486c-a8fb-8e44eadfd1fe:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 fece79db-06b8-4c45-9c04-23dceab10543:/dev/nvme0n1p3]
May 25 16:36:35.112252 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.112179    2554 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 25 16:36:35.115983 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.115966    2554 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 25 16:36:35.117320 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.117198    2554 manager.go:217] Machine: {Timestamp:2026-05-25 16:36:35.116666452 +0000 UTC m=+0.464584694 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099709 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f7b082a1ec369e7f935afef638c26 SystemUUID:ec2f7b08-2a1e-c369-e7f9-35afef638c26 BootID:504840cd-5fb9-4595-bff2-425a9d1dc292 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c4:b2:97:14:95 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c4:b2:97:14:95 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:5b:42:bc:81:4d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 25 16:36:35.117320 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.117318    2554 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 25 16:36:35.117427 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.117401    2554 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 25 16:36:35.120592 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.120567    2554 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 25 16:36:35.120744 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.120596    2554 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-137.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 25 16:36:35.120790 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.120754    2554 topology_manager.go:138] "Creating topology manager with none policy"
May 25 16:36:35.120790 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.120763    2554 container_manager_linux.go:306] "Creating device plugin manager"
May 25 16:36:35.120790 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.120771    2554 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 25 16:36:35.122463 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.122452    2554 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 25 16:36:35.125275 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.125262    2554 state_mem.go:36] "Initialized new in-memory state store"
May 25 16:36:35.125386 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.125377    2554 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 25 16:36:35.128025 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.128015    2554 kubelet.go:491] "Attempting to sync node with API server"
May 25 16:36:35.128068 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.128029    2554 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 25 16:36:35.128068 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.128043    2554 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 25 16:36:35.128068 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.128053    2554 kubelet.go:397] "Adding apiserver pod source"
May 25 16:36:35.128068 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.128061    2554 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 25 16:36:35.129308 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.129294    2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 25 16:36:35.129366 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.129314    2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 25 16:36:35.132781 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.132756    2554 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 25 16:36:35.135055 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.135041    2554 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 25 16:36:35.136031 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.136020    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 25 16:36:35.136085 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.136036    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 25 16:36:35.136085 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.136043    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 25 16:36:35.136085 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.136048    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 25 16:36:35.136085 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.136058    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 25 16:36:35.136085 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.136063    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 25 16:36:35.136085 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.136069    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 25 16:36:35.136085 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.136075    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 25 16:36:35.136281 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.136091    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 25 16:36:35.136281 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.136097    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 25 16:36:35.136281 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.136112    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 25 16:36:35.136537 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.136527    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 25 16:36:35.136568 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.136557    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 25 16:36:35.136568 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.136562    2554 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 25 16:36:35.140056 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.140043    2554 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 25 16:36:35.140138 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.140080    2554 server.go:1295] "Started kubelet"
May 25 16:36:35.140180 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.140135    2554 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 25 16:36:35.140256 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.140182    2554 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 25 16:36:35.140292 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.140279    2554 server_v1.go:47] "podresources" method="list" useActivePods=true
May 25 16:36:35.140993 ip-10-0-135-137 systemd[1]: Started Kubernetes Kubelet.
May 25 16:36:35.141938 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.141921    2554 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 25 16:36:35.143124 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.143086    2554 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-137.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 25 16:36:35.143311 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.143285    2554 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-137.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 25 16:36:35.143373 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.143310    2554 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 25 16:36:35.143470 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.143459    2554 server.go:317] "Adding debug handlers to kubelet server"
May 25 16:36:35.146998 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.145886    2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-137.ec2.internal.18b2dc45ce7f11be  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-137.ec2.internal,UID:ip-10-0-135-137.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-137.ec2.internal,},FirstTimestamp:2026-05-25 16:36:35.140055486 +0000 UTC m=+0.487973729,LastTimestamp:2026-05-25 16:36:35.140055486 +0000 UTC m=+0.487973729,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-137.ec2.internal,}"
May 25 16:36:35.150945 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.150923    2554 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 25 16:36:35.151045 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.150942    2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 25 16:36:35.151045 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.151006    2554 volume_manager.go:295] "The desired_state_of_world populator starts"
May 25 16:36:35.151045 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.151016    2554 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 25 16:36:35.151189 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.151047    2554 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 25 16:36:35.151189 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.151180    2554 reconstruct.go:97] "Volume reconstruction finished"
May 25 16:36:35.151189 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.151188    2554 reconciler.go:26] "Reconciler: start to sync state"
May 25 16:36:35.151641 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.151623    2554 factory.go:55] Registering systemd factory
May 25 16:36:35.151641 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.151643    2554 factory.go:223] Registration of the systemd container factory successfully
May 25 16:36:35.151824 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.151803    2554 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 25 16:36:35.151908 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.151852    2554 factory.go:153] Registering CRI-O factory
May 25 16:36:35.151908 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.151882    2554 factory.go:223] Registration of the crio container factory successfully
May 25 16:36:35.151983 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.151935    2554 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 25 16:36:35.151983 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.151936    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-137.ec2.internal\" not found"
May 25 16:36:35.151983 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.151957    2554 factory.go:103] Registering Raw factory
May 25 16:36:35.151983 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.151971    2554 manager.go:1196] Started watching for new ooms in manager
May 25 16:36:35.153232 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.153179    2554 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-137.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
May 25 16:36:35.153380 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.153353    2554 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
May 25 16:36:35.153512 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.153495    2554 manager.go:319] Starting recovery of all containers
May 25 16:36:35.162710 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.162543    2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9xqbv"
May 25 16:36:35.164370 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.164351    2554 manager.go:324] Recovery completed
May 25 16:36:35.165779 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.165760    2554 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
May 25 16:36:35.168722 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.168709    2554 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 16:36:35.170446 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.170427    2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9xqbv"
May 25 16:36:35.172254 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.172236    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasSufficientMemory"
May 25 16:36:35.172325 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.172272    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasNoDiskPressure"
May 25 16:36:35.172325 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.172283    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasSufficientPID"
May 25 16:36:35.172763 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.172748    2554 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 25 16:36:35.172763 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.172760    2554 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 25 16:36:35.172906 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.172775    2554 state_mem.go:36] "Initialized new in-memory state store"
May 25 16:36:35.176661 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.176649    2554 policy_none.go:49] "None policy: Start"
May 25 16:36:35.176731 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.176666    2554 memory_manager.go:186] "Starting memorymanager" policy="None"
May 25 16:36:35.176731 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.176677    2554 state_mem.go:35] "Initializing new in-memory state store"
May 25 16:36:35.213571 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.213552    2554 manager.go:341] "Starting Device Plugin manager"
May 25 16:36:35.213696 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.213593    2554 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 25 16:36:35.213696 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.213607    2554 server.go:85] "Starting device plugin registration server"
May 25 16:36:35.214104 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.214086    2554 eviction_manager.go:189] "Eviction manager: starting control loop"
May 25 16:36:35.214209 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.214105    2554 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 25 16:36:35.214274 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.214258    2554 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 25 16:36:35.214389 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.214330    2554 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 25 16:36:35.214389 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.214339    2554 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 25 16:36:35.214950 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.214856    2554 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 25 16:36:35.214950 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.214918    2554 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-137.ec2.internal\" not found"
May 25 16:36:35.260222 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.260188    2554 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 25 16:36:35.264474 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.261487    2554 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 25 16:36:35.264474 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.261514    2554 status_manager.go:230] "Starting to sync pod status with apiserver"
May 25 16:36:35.264474 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.261539    2554 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 25 16:36:35.264474 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.261547    2554 kubelet.go:2451] "Starting kubelet main sync loop"
May 25 16:36:35.264474 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.261589    2554 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 25 16:36:35.264474 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.264270    2554 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 16:36:35.314647 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.314571    2554 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 16:36:35.315565 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.315548    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasSufficientMemory"
May 25 16:36:35.315638 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.315579    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasNoDiskPressure"
May 25 16:36:35.315638 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.315592    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasSufficientPID"
May 25 16:36:35.315638 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.315623    2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-137.ec2.internal"
May 25 16:36:35.322862 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.322844    2554 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-137.ec2.internal"
May 25 16:36:35.322940 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.322883    2554 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-137.ec2.internal\": node \"ip-10-0-135-137.ec2.internal\" not found"
May 25 16:36:35.335371 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.335349    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-137.ec2.internal\" not found"
May 25 16:36:35.362474 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.362445    2554 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-137.ec2.internal"]
May 25 16:36:35.362613 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.362525    2554 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 16:36:35.363715 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.363700    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasSufficientMemory"
May 25 16:36:35.363817 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.363730    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasNoDiskPressure"
May 25 16:36:35.363817 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.363744    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasSufficientPID"
May 25 16:36:35.365250 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.365236    2554 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 16:36:35.365383 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.365367    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal"
May 25 16:36:35.365423 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.365402    2554 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 16:36:35.365986 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.365969    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasSufficientMemory"
May 25 16:36:35.365986 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.365977    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasSufficientMemory"
May 25 16:36:35.366135 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.365998    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasNoDiskPressure"
May 25 16:36:35.366135 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.366009    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasSufficientPID"
May 25 16:36:35.366135 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.365999    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasNoDiskPressure"
May 25 16:36:35.366135 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.366105    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasSufficientPID"
May 25 16:36:35.367340 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.367326    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-137.ec2.internal"
May 25 16:36:35.367402 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.367352    2554 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 16:36:35.368036 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.368021    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasSufficientMemory"
May 25 16:36:35.368122 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.368048    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasNoDiskPressure"
May 25 16:36:35.368122 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.368062    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeHasSufficientPID"
May 25 16:36:35.391710 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.391683    2554 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-137.ec2.internal\" not found" node="ip-10-0-135-137.ec2.internal"
May 25 16:36:35.396233 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.396215    2554 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-137.ec2.internal\" not found" node="ip-10-0-135-137.ec2.internal"
May 25 16:36:35.436326 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.436296    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-137.ec2.internal\" not found"
May 25 16:36:35.452564 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.452535    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6cfdc7d2df72a17445303fc2d13b0a7e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal\" (UID: \"6cfdc7d2df72a17445303fc2d13b0a7e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal"
May 25 16:36:35.452639 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.452570    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cfdc7d2df72a17445303fc2d13b0a7e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal\" (UID: \"6cfdc7d2df72a17445303fc2d13b0a7e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal"
May 25 16:36:35.452639 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.452597    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9e7d79ffc7bf90f02369e701a94a0e0a-config\") pod \"kube-apiserver-proxy-ip-10-0-135-137.ec2.internal\" (UID: \"9e7d79ffc7bf90f02369e701a94a0e0a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-137.ec2.internal"
May 25 16:36:35.536830 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.536792    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-137.ec2.internal\" not found"
May 25 16:36:35.553150 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.553127    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6cfdc7d2df72a17445303fc2d13b0a7e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal\" (UID: \"6cfdc7d2df72a17445303fc2d13b0a7e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal"
May 25 16:36:35.553202 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.553155    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cfdc7d2df72a17445303fc2d13b0a7e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal\" (UID: \"6cfdc7d2df72a17445303fc2d13b0a7e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal"
May 25 16:36:35.553202 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.553173    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9e7d79ffc7bf90f02369e701a94a0e0a-config\") pod \"kube-apiserver-proxy-ip-10-0-135-137.ec2.internal\" (UID: \"9e7d79ffc7bf90f02369e701a94a0e0a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-137.ec2.internal"
May 25 16:36:35.553521 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.553325    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cfdc7d2df72a17445303fc2d13b0a7e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal\" (UID: \"6cfdc7d2df72a17445303fc2d13b0a7e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal"
May 25 16:36:35.553662 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.553548    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6cfdc7d2df72a17445303fc2d13b0a7e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal\" (UID: \"6cfdc7d2df72a17445303fc2d13b0a7e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal"
May 25 16:36:35.553662 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.553560    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9e7d79ffc7bf90f02369e701a94a0e0a-config\") pod \"kube-apiserver-proxy-ip-10-0-135-137.ec2.internal\" (UID: \"9e7d79ffc7bf90f02369e701a94a0e0a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-137.ec2.internal"
May 25 16:36:35.637674 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.637579    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-137.ec2.internal\" not found"
May 25 16:36:35.696099 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.696058    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal"
May 25 16:36:35.698933 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.698917    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-137.ec2.internal"
May 25 16:36:35.738742 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.738713    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-137.ec2.internal\" not found"
May 25 16:36:35.839171 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.839136    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-137.ec2.internal\" not found"
May 25 16:36:35.939680 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:35.939600    2554 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-137.ec2.internal\" not found"
May 25 16:36:35.957076 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:35.957054    2554 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 16:36:36.048343 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.048315    2554 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 25 16:36:36.049020 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.048443    2554 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 25 16:36:36.049020 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.048480    2554 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 25 16:36:36.051480 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.051459    2554 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal"
May 25 16:36:36.076813 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.076791    2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 25 16:36:36.078279 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.078266    2554 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-137.ec2.internal"
May 25 16:36:36.085285 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.085265    2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 25 16:36:36.128482 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.128428    2554 apiserver.go:52] "Watching apiserver"
May 25 16:36:36.138105 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.138080    2554 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 25 16:36:36.138530 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.138502    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-l57q2","openshift-network-operator/iptables-alerter-7klm9","openshift-ovn-kubernetes/ovnkube-node-ggvbg","kube-system/global-pull-secret-syncer-bdqw9","kube-system/konnectivity-agent-xfhkv","kube-system/kube-apiserver-proxy-ip-10-0-135-137.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888","openshift-cluster-node-tuning-operator/tuned-vx8bb","openshift-image-registry/node-ca-wp4m6","openshift-multus/multus-additional-cni-plugins-n7gn7","openshift-network-diagnostics/network-check-target-fxfkl","openshift-dns/node-resolver-bx8kp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal","openshift-multus/multus-nbfsz"]
May 25 16:36:36.142614 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.142267    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.144257 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.144235    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:36.144372 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.144319    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:36:36.145455 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.145437    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 25 16:36:36.146125 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.145999    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-62qx4\""
May 25 16:36:36.146125 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.146068    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 25 16:36:36.146289 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.146275    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 25 16:36:36.146580 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.146566    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.148539 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.148520    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:36.148628 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.148581    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:36:36.149811 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.149793    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 25 16:36:36.149917 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.149876    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 25 16:36:36.149993 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.149978    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 25 16:36:36.150044 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.150029    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 25 16:36:36.150128 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.150113    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 25 16:36:36.150481 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.150461    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 25 16:36:36.150481 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.150475    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q4cd8\""
May 25 16:36:36.150780 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.150767    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xfhkv"
May 25 16:36:36.151063 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.151045    2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 25 16:36:36.153212 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.153191    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.155470 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.155449    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wp4m6"
May 25 16:36:36.155559 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.155483    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7n69\" (UniqueName: \"kubernetes.io/projected/230e7d42-ab66-49b4-876c-969beac2ebca-kube-api-access-s7n69\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.155559 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.155515    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/eb80c465-3f53-4d73-abef-d8465200fa2c-kubelet-config\") pod \"global-pull-secret-syncer-bdqw9\" (UID: \"eb80c465-3f53-4d73-abef-d8465200fa2c\") " pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:36.155559 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.155542    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret\") pod \"global-pull-secret-syncer-bdqw9\" (UID: \"eb80c465-3f53-4d73-abef-d8465200fa2c\") " pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:36.155704 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.155567    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-multus-cni-dir\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.155704 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.155612    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-sys-fs\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.155704 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.155638    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-run-ovn-kubernetes\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.155704 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.155663    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-os-release\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.155704 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.155688    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-multus-socket-dir-parent\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.156268 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156047    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-run-multus-certs\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.156268 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156124    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/230e7d42-ab66-49b4-876c-969beac2ebca-env-overrides\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.156268 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156160    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/230e7d42-ab66-49b4-876c-969beac2ebca-ovn-node-metrics-cert\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.156268 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156195    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/eb80c465-3f53-4d73-abef-d8465200fa2c-dbus\") pod \"global-pull-secret-syncer-bdqw9\" (UID: \"eb80c465-3f53-4d73-abef-d8465200fa2c\") " pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:36.156268 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156240    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1018b970-9e3e-4eb4-a045-17340c9ea537-agent-certs\") pod \"konnectivity-agent-xfhkv\" (UID: \"1018b970-9e3e-4eb4-a045-17340c9ea537\") " pod="kube-system/konnectivity-agent-xfhkv"
May 25 16:36:36.156679 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156279    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1018b970-9e3e-4eb4-a045-17340c9ea537-konnectivity-ca\") pod \"konnectivity-agent-xfhkv\" (UID: \"1018b970-9e3e-4eb4-a045-17340c9ea537\") " pod="kube-system/konnectivity-agent-xfhkv"
May 25 16:36:36.156679 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156313    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-cnibin\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.156679 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156394    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e86de1e-2942-4e72-b2e7-5243b02ed554-cni-binary-copy\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.156679 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156448    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-var-lib-kubelet\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.156679 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156499    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-hostroot\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.156679 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156567    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-multus-conf-dir\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.156679 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156611    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-etc-kubernetes\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.156679 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156649    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-run-systemd\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.156679 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156675    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-node-log\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.157102 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156729    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/230e7d42-ab66-49b4-876c-969beac2ebca-ovnkube-script-lib\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.157102 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156759    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-system-cni-dir\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.157102 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156778    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-run-k8s-cni-cncf-io\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.157102 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156856    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-run-netns\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.157102 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156898    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfkrq\" (UniqueName: \"kubernetes.io/projected/9e86de1e-2942-4e72-b2e7-5243b02ed554-kube-api-access-vfkrq\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.157102 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156938    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-device-dir\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.157102 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.156968    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnn7q\" (UniqueName: \"kubernetes.io/projected/77ec7474-5075-48b6-8fba-b37947ad9c5a-kube-api-access-hnn7q\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.157102 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157025    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-etc-openvswitch\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.157102 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157064    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9e86de1e-2942-4e72-b2e7-5243b02ed554-multus-daemon-config\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.157102 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157097    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.157519 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157122    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-kubelet\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.157519 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157151    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-run-netns\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.157519 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157180    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-var-lib-openvswitch\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.157519 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157210    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-cni-netd\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.157519 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157249    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-var-lib-cni-bin\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.157519 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157274    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-var-lib-cni-multus\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.157519 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157303    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-socket-dir\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.157519 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157333    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-registration-dir\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.157519 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157395    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-etc-selinux\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.157519 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157441    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7tfc\" (UniqueName: \"kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc\") pod \"network-check-target-fxfkl\" (UID: \"11062d63-8afe-498a-8186-ba3987ab73d4\") " pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:36.157519 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157493    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-systemd-units\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.158004 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157525    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-run-openvswitch\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.158004 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157583    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-run-ovn\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.158004 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157616    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.158004 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157645    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-slash\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.158004 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157687    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-log-socket\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.158004 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157749    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-cni-bin\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.158004 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.157780    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/230e7d42-ab66-49b4-876c-969beac2ebca-ovnkube-config\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.159171 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.159153    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.159937 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.159916    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 25 16:36:36.160010 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.159970    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 25 16:36:36.160010 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.159970    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 25 16:36:36.160082 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.160042    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 25 16:36:36.160117 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.160083    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-229q4\""
May 25 16:36:36.160173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.160156    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 25 16:36:36.160173 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.160166    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 25 16:36:36.160249 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.160157    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 25 16:36:36.160402 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.160388    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 25 16:36:36.160448 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.160416    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kk9t8\""
May 25 16:36:36.160938 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.160926    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 25 16:36:36.161608 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.161594    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7klm9"
May 25 16:36:36.162454 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.162441    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jfjwr\""
May 25 16:36:36.163721 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.163706    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.165144 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.165129    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5vqgm\""
May 25 16:36:36.165768 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.165755    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:36.165849 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.165810    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:36:36.167952 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.167935    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bx8kp"
May 25 16:36:36.170978 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.170961    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 25 16:36:36.172327 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.172290    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-24 16:31:35 +0000 UTC" deadline="2027-11-29 21:47:25.832973503 +0000 UTC"
May 25 16:36:36.172397 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.172329    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13277h10m49.660646743s"
May 25 16:36:36.172397 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.172315    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 25 16:36:36.174221 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.174206    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 25 16:36:36.175532 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.175512    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2ts4x\""
May 25 16:36:36.175607 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.175556    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 25 16:36:36.176476 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.176459    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 25 16:36:36.178007 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.177992    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 25 16:36:36.178975 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.178961    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 25 16:36:36.179694 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.179681    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 25 16:36:36.183811 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.183794    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-b5xjv\""
May 25 16:36:36.183914 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.183827    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 25 16:36:36.184199 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.184175    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-w9q2w\""
May 25 16:36:36.189155 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.189127    2554 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 25 16:36:36.252723 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.252697    2554 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 25 16:36:36.258333 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258314    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-run-k8s-cni-cncf-io\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.258409 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258342    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-run-netns\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.258409 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258361    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vfkrq\" (UniqueName: \"kubernetes.io/projected/9e86de1e-2942-4e72-b2e7-5243b02ed554-kube-api-access-vfkrq\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.258409 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258385    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-systemd\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.258529 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258411    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6cc91204-a4f1-4108-b4d6-dab40dbda044-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.258529 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258429    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-run-k8s-cni-cncf-io\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.258529 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258439    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hnn7q\" (UniqueName: \"kubernetes.io/projected/77ec7474-5075-48b6-8fba-b37947ad9c5a-kube-api-access-hnn7q\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.258529 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258464    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-etc-openvswitch\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.258529 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258458    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-run-netns\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.258736 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258562    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6cc91204-a4f1-4108-b4d6-dab40dbda044-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.258736 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258589    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-etc-openvswitch\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.258736 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258593    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-run-netns\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.258736 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258627    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-run-netns\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.258736 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258718    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-cni-netd\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.258736 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258736    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-host\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.258992 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258751    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-socket-dir\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.258992 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258765    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-registration-dir\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.258992 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258782    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g7tfc\" (UniqueName: \"kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc\") pod \"network-check-target-fxfkl\" (UID: \"11062d63-8afe-498a-8186-ba3987ab73d4\") " pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:36.258992 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258812    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-systemd-units\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.258992 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258846    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-systemd-units\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.258992 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258888    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-run-ovn\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.258992 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258913    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-cni-netd\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.258992 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258953    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-run-ovn\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.258992 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258962    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-socket-dir\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.258992 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258964    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-registration-dir\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.258992 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.258988    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259011    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/645b9dd9-5a8a-406b-a2af-5832530e4664-tmp-dir\") pod \"node-resolver-bx8kp\" (UID: \"645b9dd9-5a8a-406b-a2af-5832530e4664\") " pod="openshift-dns/node-resolver-bx8kp"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259026    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-os-release\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259067    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259095    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-run-multus-certs\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259068    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-run-multus-certs\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259128    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d15b78cb-69ab-4017-8bae-92669f025b54-serviceca\") pod \"node-ca-wp4m6\" (UID: \"d15b78cb-69ab-4017-8bae-92669f025b54\") " pod="openshift-image-registry/node-ca-wp4m6"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259145    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-slash\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259161    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-log-socket\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259169    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-os-release\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259184    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s7n69\" (UniqueName: \"kubernetes.io/projected/230e7d42-ab66-49b4-876c-969beac2ebca-kube-api-access-s7n69\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259203    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-slash\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259223    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/eb80c465-3f53-4d73-abef-d8465200fa2c-kubelet-config\") pod \"global-pull-secret-syncer-bdqw9\" (UID: \"eb80c465-3f53-4d73-abef-d8465200fa2c\") " pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259254    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret\") pod \"global-pull-secret-syncer-bdqw9\" (UID: \"eb80c465-3f53-4d73-abef-d8465200fa2c\") " pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259285    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msflv\" (UniqueName: \"kubernetes.io/projected/d15b78cb-69ab-4017-8bae-92669f025b54-kube-api-access-msflv\") pod \"node-ca-wp4m6\" (UID: \"d15b78cb-69ab-4017-8bae-92669f025b54\") " pod="openshift-image-registry/node-ca-wp4m6"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259257    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-log-socket\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259315    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-multus-cni-dir\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.259445 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259320    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/eb80c465-3f53-4d73-abef-d8465200fa2c-kubelet-config\") pod \"global-pull-secret-syncer-bdqw9\" (UID: \"eb80c465-3f53-4d73-abef-d8465200fa2c\") " pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259343    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-sysctl-conf\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.259359    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259369    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-sys\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259380    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-multus-cni-dir\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259394    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-var-lib-kubelet\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.259429    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret podName:eb80c465-3f53-4d73-abef-d8465200fa2c nodeName:}" failed. No retries permitted until 2026-05-25 16:36:36.759406484 +0000 UTC m=+2.107324733 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret") pod "global-pull-secret-syncer-bdqw9" (UID: "eb80c465-3f53-4d73-abef-d8465200fa2c") : object "kube-system"/"original-pull-secret" not registered
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259452    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lbgr\" (UniqueName: \"kubernetes.io/projected/d4f109bb-cd5d-47ea-a8d1-af800dc99307-kube-api-access-6lbgr\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259481    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgt9z\" (UniqueName: \"kubernetes.io/projected/f603e2fb-4fa8-4746-a721-fb1e1342fc6b-kube-api-access-bgt9z\") pod \"iptables-alerter-7klm9\" (UID: \"f603e2fb-4fa8-4746-a721-fb1e1342fc6b\") " pod="openshift-network-operator/iptables-alerter-7klm9"
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259507    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cc91204-a4f1-4108-b4d6-dab40dbda044-cni-binary-copy\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259532    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/230e7d42-ab66-49b4-876c-969beac2ebca-ovn-node-metrics-cert\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259561    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/eb80c465-3f53-4d73-abef-d8465200fa2c-dbus\") pod \"global-pull-secret-syncer-bdqw9\" (UID: \"eb80c465-3f53-4d73-abef-d8465200fa2c\") " pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259587    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1018b970-9e3e-4eb4-a045-17340c9ea537-konnectivity-ca\") pod \"konnectivity-agent-xfhkv\" (UID: \"1018b970-9e3e-4eb4-a045-17340c9ea537\") " pod="kube-system/konnectivity-agent-xfhkv"
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259611    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-cnibin\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259634    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-var-lib-kubelet\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259663    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-cnibin\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259680    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-var-lib-kubelet\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.260204 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259698    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/eb80c465-3f53-4d73-abef-d8465200fa2c-dbus\") pod \"global-pull-secret-syncer-bdqw9\" (UID: \"eb80c465-3f53-4d73-abef-d8465200fa2c\") " pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259785    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-hostroot\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259816    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-etc-kubernetes\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259830    2554 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259842    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-modprobe-d\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259885    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d15b78cb-69ab-4017-8bae-92669f025b54-host\") pod \"node-ca-wp4m6\" (UID: \"d15b78cb-69ab-4017-8bae-92669f025b54\") " pod="openshift-image-registry/node-ca-wp4m6"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259892    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-hostroot\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259910    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-sysctl-d\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259941    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzjrs\" (UniqueName: \"kubernetes.io/projected/6cc91204-a4f1-4108-b4d6-dab40dbda044-kube-api-access-lzjrs\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259960    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-etc-kubernetes\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259966    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-run-systemd\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.259993    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-node-log\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260005    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-run-systemd\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260018    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/230e7d42-ab66-49b4-876c-969beac2ebca-ovnkube-script-lib\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260048    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-system-cni-dir\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260058    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-node-log\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260071    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-sysconfig\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260123    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-system-cni-dir\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.260974 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260151    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-kubernetes\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260173    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1018b970-9e3e-4eb4-a045-17340c9ea537-konnectivity-ca\") pod \"konnectivity-agent-xfhkv\" (UID: \"1018b970-9e3e-4eb4-a045-17340c9ea537\") " pod="kube-system/konnectivity-agent-xfhkv"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260176    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6cc91204-a4f1-4108-b4d6-dab40dbda044-os-release\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260202    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-device-dir\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260227    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9e86de1e-2942-4e72-b2e7-5243b02ed554-multus-daemon-config\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260248    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/645b9dd9-5a8a-406b-a2af-5832530e4664-hosts-file\") pod \"node-resolver-bx8kp\" (UID: \"645b9dd9-5a8a-406b-a2af-5832530e4664\") " pod="openshift-dns/node-resolver-bx8kp"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260269    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260290    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-kubelet\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260312    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-var-lib-openvswitch\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260332    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260365    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-kubelet\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260266    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-device-dir\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260374    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-var-lib-cni-bin\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260334    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-var-lib-cni-bin\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260413    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-var-lib-cni-multus\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260415    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-var-lib-openvswitch\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260439    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-etc-selinux\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.261683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260464    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-run-openvswitch\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260480    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-host-var-lib-cni-multus\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260489    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/230e7d42-ab66-49b4-876c-969beac2ebca-ovnkube-config\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260517    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-multus-socket-dir-parent\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260538    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-etc-selinux\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260543    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-cni-bin\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260556    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-run-openvswitch\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260583    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-cni-bin\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260586    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f603e2fb-4fa8-4746-a721-fb1e1342fc6b-iptables-alerter-script\") pod \"iptables-alerter-7klm9\" (UID: \"f603e2fb-4fa8-4746-a721-fb1e1342fc6b\") " pod="openshift-network-operator/iptables-alerter-7klm9"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260613    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/230e7d42-ab66-49b4-876c-969beac2ebca-ovnkube-script-lib\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260624    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs\") pod \"network-metrics-daemon-l57q2\" (UID: \"6646a861-65d0-4b6f-af0c-415ff46851dd\") " pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260634    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-multus-socket-dir-parent\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260666    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-sys-fs\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260711    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-run-ovn-kubernetes\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260741    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-lib-modules\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260712    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/77ec7474-5075-48b6-8fba-b37947ad9c5a-sys-fs\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260766    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4f109bb-cd5d-47ea-a8d1-af800dc99307-tmp\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.262168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260766    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9e86de1e-2942-4e72-b2e7-5243b02ed554-multus-daemon-config\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260792    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/230e7d42-ab66-49b4-876c-969beac2ebca-host-run-ovn-kubernetes\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260806    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cc91204-a4f1-4108-b4d6-dab40dbda044-system-cni-dir\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260836    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cc91204-a4f1-4108-b4d6-dab40dbda044-cnibin\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260915    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cc91204-a4f1-4108-b4d6-dab40dbda044-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260945    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/230e7d42-ab66-49b4-876c-969beac2ebca-env-overrides\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260970    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1018b970-9e3e-4eb4-a045-17340c9ea537-agent-certs\") pod \"konnectivity-agent-xfhkv\" (UID: \"1018b970-9e3e-4eb4-a045-17340c9ea537\") " pod="kube-system/konnectivity-agent-xfhkv"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.260994    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e86de1e-2942-4e72-b2e7-5243b02ed554-cni-binary-copy\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.261019    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-tuned\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.261044    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgqhx\" (UniqueName: \"kubernetes.io/projected/6646a861-65d0-4b6f-af0c-415ff46851dd-kube-api-access-tgqhx\") pod \"network-metrics-daemon-l57q2\" (UID: \"6646a861-65d0-4b6f-af0c-415ff46851dd\") " pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.261070    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/230e7d42-ab66-49b4-876c-969beac2ebca-ovnkube-config\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.261071    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-multus-conf-dir\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.261099    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbcdv\" (UniqueName: \"kubernetes.io/projected/645b9dd9-5a8a-406b-a2af-5832530e4664-kube-api-access-hbcdv\") pod \"node-resolver-bx8kp\" (UID: \"645b9dd9-5a8a-406b-a2af-5832530e4664\") " pod="openshift-dns/node-resolver-bx8kp"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.261123    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-run\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.261161    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f603e2fb-4fa8-4746-a721-fb1e1342fc6b-host-slash\") pod \"iptables-alerter-7klm9\" (UID: \"f603e2fb-4fa8-4746-a721-fb1e1342fc6b\") " pod="openshift-network-operator/iptables-alerter-7klm9"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.261265    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e86de1e-2942-4e72-b2e7-5243b02ed554-multus-conf-dir\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.261383    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/230e7d42-ab66-49b4-876c-969beac2ebca-env-overrides\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.261560    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e86de1e-2942-4e72-b2e7-5243b02ed554-cni-binary-copy\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.262555    2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hf97q"
May 25 16:36:36.263616 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.262979    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/230e7d42-ab66-49b4-876c-969beac2ebca-ovn-node-metrics-cert\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.263616 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.263334    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1018b970-9e3e-4eb4-a045-17340c9ea537-agent-certs\") pod \"konnectivity-agent-xfhkv\" (UID: \"1018b970-9e3e-4eb4-a045-17340c9ea537\") " pod="kube-system/konnectivity-agent-xfhkv"
May 25 16:36:36.267254 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.267232    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 16:36:36.267363 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.267260    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 16:36:36.267363 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.267274    2554 projected.go:194] Error preparing data for projected volume kube-api-access-g7tfc for pod openshift-network-diagnostics/network-check-target-fxfkl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 16:36:36.267363 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.267331    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc podName:11062d63-8afe-498a-8186-ba3987ab73d4 nodeName:}" failed. No retries permitted until 2026-05-25 16:36:36.76731393 +0000 UTC m=+2.115232182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-g7tfc" (UniqueName: "kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc") pod "network-check-target-fxfkl" (UID: "11062d63-8afe-498a-8186-ba3987ab73d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 16:36:36.267363 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:36.267336    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e7d79ffc7bf90f02369e701a94a0e0a.slice/crio-118bf1f96212654336ffdb1e2bcf67f0996dbc0bd9c0dace6a5d2a2cbc99e04e WatchSource:0}: Error finding container 118bf1f96212654336ffdb1e2bcf67f0996dbc0bd9c0dace6a5d2a2cbc99e04e: Status 404 returned error can't find the container with id 118bf1f96212654336ffdb1e2bcf67f0996dbc0bd9c0dace6a5d2a2cbc99e04e
May 25 16:36:36.267579 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:36.267566    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cfdc7d2df72a17445303fc2d13b0a7e.slice/crio-b66653f581cd3d0899597e88e4fa371fd6556e9b1a8c70635a876f8ca84229df WatchSource:0}: Error finding container b66653f581cd3d0899597e88e4fa371fd6556e9b1a8c70635a876f8ca84229df: Status 404 returned error can't find the container with id b66653f581cd3d0899597e88e4fa371fd6556e9b1a8c70635a876f8ca84229df
May 25 16:36:36.268463 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.268435    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfkrq\" (UniqueName: \"kubernetes.io/projected/9e86de1e-2942-4e72-b2e7-5243b02ed554-kube-api-access-vfkrq\") pod \"multus-nbfsz\" (UID: \"9e86de1e-2942-4e72-b2e7-5243b02ed554\") " pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.269383 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.269362    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnn7q\" (UniqueName: \"kubernetes.io/projected/77ec7474-5075-48b6-8fba-b37947ad9c5a-kube-api-access-hnn7q\") pod \"aws-ebs-csi-driver-node-76888\" (UID: \"77ec7474-5075-48b6-8fba-b37947ad9c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.270788 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.270769    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7n69\" (UniqueName: \"kubernetes.io/projected/230e7d42-ab66-49b4-876c-969beac2ebca-kube-api-access-s7n69\") pod \"ovnkube-node-ggvbg\" (UID: \"230e7d42-ab66-49b4-876c-969beac2ebca\") " pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.271318 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.271302    2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hf97q"
May 25 16:36:36.272133 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.272119    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 16:36:36.316532 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.316512    2554 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 16:36:36.361843 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.361811    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-lib-modules\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.361843 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.361841    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4f109bb-cd5d-47ea-a8d1-af800dc99307-tmp\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.362084 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.361859    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cc91204-a4f1-4108-b4d6-dab40dbda044-system-cni-dir\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.362084 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.361889    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cc91204-a4f1-4108-b4d6-dab40dbda044-cnibin\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.362084 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.361905    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cc91204-a4f1-4108-b4d6-dab40dbda044-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.362084 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.361929    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-tuned\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.362084 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.361956    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tgqhx\" (UniqueName: \"kubernetes.io/projected/6646a861-65d0-4b6f-af0c-415ff46851dd-kube-api-access-tgqhx\") pod \"network-metrics-daemon-l57q2\" (UID: \"6646a861-65d0-4b6f-af0c-415ff46851dd\") " pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:36.362084 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.361968    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cc91204-a4f1-4108-b4d6-dab40dbda044-cnibin\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.362084 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.361953    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cc91204-a4f1-4108-b4d6-dab40dbda044-system-cni-dir\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.362084 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.361995    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-lib-modules\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.362084 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.361987    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hbcdv\" (UniqueName: \"kubernetes.io/projected/645b9dd9-5a8a-406b-a2af-5832530e4664-kube-api-access-hbcdv\") pod \"node-resolver-bx8kp\" (UID: \"645b9dd9-5a8a-406b-a2af-5832530e4664\") " pod="openshift-dns/node-resolver-bx8kp"
May 25 16:36:36.362483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362108    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cc91204-a4f1-4108-b4d6-dab40dbda044-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.362483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362144    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-run\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.362483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362165    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f603e2fb-4fa8-4746-a721-fb1e1342fc6b-host-slash\") pod \"iptables-alerter-7klm9\" (UID: \"f603e2fb-4fa8-4746-a721-fb1e1342fc6b\") " pod="openshift-network-operator/iptables-alerter-7klm9"
May 25 16:36:36.362483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362188    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-systemd\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.362483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362213    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6cc91204-a4f1-4108-b4d6-dab40dbda044-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.362483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362226    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-run\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.362483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362226    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f603e2fb-4fa8-4746-a721-fb1e1342fc6b-host-slash\") pod \"iptables-alerter-7klm9\" (UID: \"f603e2fb-4fa8-4746-a721-fb1e1342fc6b\") " pod="openshift-network-operator/iptables-alerter-7klm9"
May 25 16:36:36.362483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362263    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6cc91204-a4f1-4108-b4d6-dab40dbda044-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.362483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362271    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-systemd\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.362483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362293    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-host\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.362483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362337    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/645b9dd9-5a8a-406b-a2af-5832530e4664-tmp-dir\") pod \"node-resolver-bx8kp\" (UID: \"645b9dd9-5a8a-406b-a2af-5832530e4664\") " pod="openshift-dns/node-resolver-bx8kp"
May 25 16:36:36.362483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362362    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d15b78cb-69ab-4017-8bae-92669f025b54-serviceca\") pod \"node-ca-wp4m6\" (UID: \"d15b78cb-69ab-4017-8bae-92669f025b54\") " pod="openshift-image-registry/node-ca-wp4m6"
May 25 16:36:36.362483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362354    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-host\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.362483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362401    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-msflv\" (UniqueName: \"kubernetes.io/projected/d15b78cb-69ab-4017-8bae-92669f025b54-kube-api-access-msflv\") pod \"node-ca-wp4m6\" (UID: \"d15b78cb-69ab-4017-8bae-92669f025b54\") " pod="openshift-image-registry/node-ca-wp4m6"
May 25 16:36:36.362483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362439    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-sysctl-conf\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.362483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362465    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-sys\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362498    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-var-lib-kubelet\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362526    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6lbgr\" (UniqueName: \"kubernetes.io/projected/d4f109bb-cd5d-47ea-a8d1-af800dc99307-kube-api-access-6lbgr\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362549    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bgt9z\" (UniqueName: \"kubernetes.io/projected/f603e2fb-4fa8-4746-a721-fb1e1342fc6b-kube-api-access-bgt9z\") pod \"iptables-alerter-7klm9\" (UID: \"f603e2fb-4fa8-4746-a721-fb1e1342fc6b\") " pod="openshift-network-operator/iptables-alerter-7klm9"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362577    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cc91204-a4f1-4108-b4d6-dab40dbda044-cni-binary-copy\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362608    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-modprobe-d\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362636    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d15b78cb-69ab-4017-8bae-92669f025b54-host\") pod \"node-ca-wp4m6\" (UID: \"d15b78cb-69ab-4017-8bae-92669f025b54\") " pod="openshift-image-registry/node-ca-wp4m6"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362647    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/645b9dd9-5a8a-406b-a2af-5832530e4664-tmp-dir\") pod \"node-resolver-bx8kp\" (UID: \"645b9dd9-5a8a-406b-a2af-5832530e4664\") " pod="openshift-dns/node-resolver-bx8kp"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362658    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-sysctl-d\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362684    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lzjrs\" (UniqueName: \"kubernetes.io/projected/6cc91204-a4f1-4108-b4d6-dab40dbda044-kube-api-access-lzjrs\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362723    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-sysconfig\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362747    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-kubernetes\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362770    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6cc91204-a4f1-4108-b4d6-dab40dbda044-os-release\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362797    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/645b9dd9-5a8a-406b-a2af-5832530e4664-hosts-file\") pod \"node-resolver-bx8kp\" (UID: \"645b9dd9-5a8a-406b-a2af-5832530e4664\") " pod="openshift-dns/node-resolver-bx8kp"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362816    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6cc91204-a4f1-4108-b4d6-dab40dbda044-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362842    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6cc91204-a4f1-4108-b4d6-dab40dbda044-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362948    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d15b78cb-69ab-4017-8bae-92669f025b54-serviceca\") pod \"node-ca-wp4m6\" (UID: \"d15b78cb-69ab-4017-8bae-92669f025b54\") " pod="openshift-image-registry/node-ca-wp4m6"
May 25 16:36:36.363223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362954    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-sys\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.364001 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.363154    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-modprobe-d\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.364001 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.363006    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-kubernetes\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.364001 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.363051    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-sysconfig\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.364001 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.363044    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6cc91204-a4f1-4108-b4d6-dab40dbda044-os-release\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.364001 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.363090    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-var-lib-kubelet\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.364001 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.363093    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/645b9dd9-5a8a-406b-a2af-5832530e4664-hosts-file\") pod \"node-resolver-bx8kp\" (UID: \"645b9dd9-5a8a-406b-a2af-5832530e4664\") " pod="openshift-dns/node-resolver-bx8kp"
May 25 16:36:36.364001 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.363213    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-sysctl-d\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.364001 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.363043    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-sysctl-conf\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.364001 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.363121    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d15b78cb-69ab-4017-8bae-92669f025b54-host\") pod \"node-ca-wp4m6\" (UID: \"d15b78cb-69ab-4017-8bae-92669f025b54\") " pod="openshift-image-registry/node-ca-wp4m6"
May 25 16:36:36.364001 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.362832    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f603e2fb-4fa8-4746-a721-fb1e1342fc6b-iptables-alerter-script\") pod \"iptables-alerter-7klm9\" (UID: \"f603e2fb-4fa8-4746-a721-fb1e1342fc6b\") " pod="openshift-network-operator/iptables-alerter-7klm9"
May 25 16:36:36.364001 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.363362    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs\") pod \"network-metrics-daemon-l57q2\" (UID: \"6646a861-65d0-4b6f-af0c-415ff46851dd\") " pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:36.364001 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.363463    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 16:36:36.364001 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.363517    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cc91204-a4f1-4108-b4d6-dab40dbda044-cni-binary-copy\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.364001 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.363523    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs podName:6646a861-65d0-4b6f-af0c-415ff46851dd nodeName:}" failed. No retries permitted until 2026-05-25 16:36:36.863505146 +0000 UTC m=+2.211423390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs") pod "network-metrics-daemon-l57q2" (UID: "6646a861-65d0-4b6f-af0c-415ff46851dd") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 16:36:36.364001 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.363709    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f603e2fb-4fa8-4746-a721-fb1e1342fc6b-iptables-alerter-script\") pod \"iptables-alerter-7klm9\" (UID: \"f603e2fb-4fa8-4746-a721-fb1e1342fc6b\") " pod="openshift-network-operator/iptables-alerter-7klm9"
May 25 16:36:36.364641 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.364592    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4f109bb-cd5d-47ea-a8d1-af800dc99307-tmp\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.364677 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.364654    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d4f109bb-cd5d-47ea-a8d1-af800dc99307-etc-tuned\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.373762 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.373739    2554 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 16:36:36.378382 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.378365    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msflv\" (UniqueName: \"kubernetes.io/projected/d15b78cb-69ab-4017-8bae-92669f025b54-kube-api-access-msflv\") pod \"node-ca-wp4m6\" (UID: \"d15b78cb-69ab-4017-8bae-92669f025b54\") " pod="openshift-image-registry/node-ca-wp4m6"
May 25 16:36:36.378443 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.378372    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbcdv\" (UniqueName: \"kubernetes.io/projected/645b9dd9-5a8a-406b-a2af-5832530e4664-kube-api-access-hbcdv\") pod \"node-resolver-bx8kp\" (UID: \"645b9dd9-5a8a-406b-a2af-5832530e4664\") " pod="openshift-dns/node-resolver-bx8kp"
May 25 16:36:36.387116 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.387094    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgt9z\" (UniqueName: \"kubernetes.io/projected/f603e2fb-4fa8-4746-a721-fb1e1342fc6b-kube-api-access-bgt9z\") pod \"iptables-alerter-7klm9\" (UID: \"f603e2fb-4fa8-4746-a721-fb1e1342fc6b\") " pod="openshift-network-operator/iptables-alerter-7klm9"
May 25 16:36:36.387225 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.387201    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzjrs\" (UniqueName: \"kubernetes.io/projected/6cc91204-a4f1-4108-b4d6-dab40dbda044-kube-api-access-lzjrs\") pod \"multus-additional-cni-plugins-n7gn7\" (UID: \"6cc91204-a4f1-4108-b4d6-dab40dbda044\") " pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.387311 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.387298    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lbgr\" (UniqueName: \"kubernetes.io/projected/d4f109bb-cd5d-47ea-a8d1-af800dc99307-kube-api-access-6lbgr\") pod \"tuned-vx8bb\" (UID: \"d4f109bb-cd5d-47ea-a8d1-af800dc99307\") " pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.387444 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.387428    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgqhx\" (UniqueName: \"kubernetes.io/projected/6646a861-65d0-4b6f-af0c-415ff46851dd-kube-api-access-tgqhx\") pod \"network-metrics-daemon-l57q2\" (UID: \"6646a861-65d0-4b6f-af0c-415ff46851dd\") " pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:36.477882 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.477782    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888"
May 25 16:36:36.484891 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:36.484850    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77ec7474_5075_48b6_8fba_b37947ad9c5a.slice/crio-1854f99cbec8044ef5f2faf499d5d9055449c831b1ee801047bfee131e715cd1 WatchSource:0}: Error finding container 1854f99cbec8044ef5f2faf499d5d9055449c831b1ee801047bfee131e715cd1: Status 404 returned error can't find the container with id 1854f99cbec8044ef5f2faf499d5d9055449c831b1ee801047bfee131e715cd1
May 25 16:36:36.487858 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.487843    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:36:36.494115 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:36.494095    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod230e7d42_ab66_49b4_876c_969beac2ebca.slice/crio-046f0ba89bfe0e842787f82ede405522ff5415ab9f3234d14b563c1575bfcd8c WatchSource:0}: Error finding container 046f0ba89bfe0e842787f82ede405522ff5415ab9f3234d14b563c1575bfcd8c: Status 404 returned error can't find the container with id 046f0ba89bfe0e842787f82ede405522ff5415ab9f3234d14b563c1575bfcd8c
May 25 16:36:36.510649 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.510625    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xfhkv"
May 25 16:36:36.516409 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:36.516387    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1018b970_9e3e_4eb4_a045_17340c9ea537.slice/crio-91bb239689b091692e46b47a2380c63ebb827a63bfebe6386be7aeb7c7c51893 WatchSource:0}: Error finding container 91bb239689b091692e46b47a2380c63ebb827a63bfebe6386be7aeb7c7c51893: Status 404 returned error can't find the container with id 91bb239689b091692e46b47a2380c63ebb827a63bfebe6386be7aeb7c7c51893
May 25 16:36:36.529774 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.529758    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nbfsz"
May 25 16:36:36.538589 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:36.538566    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e86de1e_2942_4e72_b2e7_5243b02ed554.slice/crio-0e69bb84ed9e155387e73d1cb3f71aee133a68ed41d67756813e1017d73cdd7b WatchSource:0}: Error finding container 0e69bb84ed9e155387e73d1cb3f71aee133a68ed41d67756813e1017d73cdd7b: Status 404 returned error can't find the container with id 0e69bb84ed9e155387e73d1cb3f71aee133a68ed41d67756813e1017d73cdd7b
May 25 16:36:36.546286 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.546269    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wp4m6"
May 25 16:36:36.551835 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:36.551811    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd15b78cb_69ab_4017_8bae_92669f025b54.slice/crio-6b92e72814c913836f1b2008e8c49b2015807dbd2fbb1993e391b953103acfaf WatchSource:0}: Error finding container 6b92e72814c913836f1b2008e8c49b2015807dbd2fbb1993e391b953103acfaf: Status 404 returned error can't find the container with id 6b92e72814c913836f1b2008e8c49b2015807dbd2fbb1993e391b953103acfaf
May 25 16:36:36.577311 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.577280    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vx8bb"
May 25 16:36:36.582952 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.582930    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7klm9"
May 25 16:36:36.583555 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:36.583398    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f109bb_cd5d_47ea_a8d1_af800dc99307.slice/crio-df3feffbb6041f2392d3be8ef2d594b64eba649559de81d34285b16ee9b067e4 WatchSource:0}: Error finding container df3feffbb6041f2392d3be8ef2d594b64eba649559de81d34285b16ee9b067e4: Status 404 returned error can't find the container with id df3feffbb6041f2392d3be8ef2d594b64eba649559de81d34285b16ee9b067e4
May 25 16:36:36.588810 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.588791    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n7gn7"
May 25 16:36:36.589076 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:36.589054    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf603e2fb_4fa8_4746_a721_fb1e1342fc6b.slice/crio-23aea757bcc1cc73422246b6e3bd7c361ecc671a7661209f2498fa0b5a7a85b4 WatchSource:0}: Error finding container 23aea757bcc1cc73422246b6e3bd7c361ecc671a7661209f2498fa0b5a7a85b4: Status 404 returned error can't find the container with id 23aea757bcc1cc73422246b6e3bd7c361ecc671a7661209f2498fa0b5a7a85b4
May 25 16:36:36.593854 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.593833    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bx8kp"
May 25 16:36:36.595916 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:36.595895    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc91204_a4f1_4108_b4d6_dab40dbda044.slice/crio-427908d24746ff9ce9b13a288357872745821c057ee7ce94a4b22808f94bb16b WatchSource:0}: Error finding container 427908d24746ff9ce9b13a288357872745821c057ee7ce94a4b22808f94bb16b: Status 404 returned error can't find the container with id 427908d24746ff9ce9b13a288357872745821c057ee7ce94a4b22808f94bb16b
May 25 16:36:36.600619 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:36:36.600597    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod645b9dd9_5a8a_406b_a2af_5832530e4664.slice/crio-da8c1ed835168fd8280a32bb7d40071904ab163e54703507538330777e663e2c WatchSource:0}: Error finding container da8c1ed835168fd8280a32bb7d40071904ab163e54703507538330777e663e2c: Status 404 returned error can't find the container with id da8c1ed835168fd8280a32bb7d40071904ab163e54703507538330777e663e2c
May 25 16:36:36.765589 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.765553    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret\") pod \"global-pull-secret-syncer-bdqw9\" (UID: \"eb80c465-3f53-4d73-abef-d8465200fa2c\") " pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:36.765745 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.765710    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 25 16:36:36.765814 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.765779    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret podName:eb80c465-3f53-4d73-abef-d8465200fa2c nodeName:}" failed. No retries permitted until 2026-05-25 16:36:37.765760547 +0000 UTC m=+3.113678800 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret") pod "global-pull-secret-syncer-bdqw9" (UID: "eb80c465-3f53-4d73-abef-d8465200fa2c") : object "kube-system"/"original-pull-secret" not registered
May 25 16:36:36.866493 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.866454    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g7tfc\" (UniqueName: \"kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc\") pod \"network-check-target-fxfkl\" (UID: \"11062d63-8afe-498a-8186-ba3987ab73d4\") " pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:36.866729 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:36.866543    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs\") pod \"network-metrics-daemon-l57q2\" (UID: \"6646a861-65d0-4b6f-af0c-415ff46851dd\") " pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:36.866729 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.866673    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 16:36:36.866729 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.866729    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs podName:6646a861-65d0-4b6f-af0c-415ff46851dd nodeName:}" failed. No retries permitted until 2026-05-25 16:36:37.866710179 +0000 UTC m=+3.214628413 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs") pod "network-metrics-daemon-l57q2" (UID: "6646a861-65d0-4b6f-af0c-415ff46851dd") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 16:36:36.867175 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.867147    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 16:36:36.867175 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.867170    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 16:36:36.867321 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.867182    2554 projected.go:194] Error preparing data for projected volume kube-api-access-g7tfc for pod openshift-network-diagnostics/network-check-target-fxfkl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 16:36:36.867321 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:36.867223    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc podName:11062d63-8afe-498a-8186-ba3987ab73d4 nodeName:}" failed. No retries permitted until 2026-05-25 16:36:37.867209413 +0000 UTC m=+3.215127659 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-g7tfc" (UniqueName: "kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc") pod "network-check-target-fxfkl" (UID: "11062d63-8afe-498a-8186-ba3987ab73d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 16:36:37.263103 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.263073    2554 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 16:36:37.263687 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.263668    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:37.263831 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:37.263806    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:36:37.272245 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.272173    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-24 16:31:36 +0000 UTC" deadline="2028-01-13 01:59:19.044740231 +0000 UTC"
May 25 16:36:37.272245 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.272204    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14337h22m41.772539877s"
May 25 16:36:37.278822 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.278700    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-137.ec2.internal" event={"ID":"9e7d79ffc7bf90f02369e701a94a0e0a","Type":"ContainerStarted","Data":"118bf1f96212654336ffdb1e2bcf67f0996dbc0bd9c0dace6a5d2a2cbc99e04e"}
May 25 16:36:37.311955 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.311856    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7klm9" event={"ID":"f603e2fb-4fa8-4746-a721-fb1e1342fc6b","Type":"ContainerStarted","Data":"23aea757bcc1cc73422246b6e3bd7c361ecc671a7661209f2498fa0b5a7a85b4"}
May 25 16:36:37.321750 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.321712    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888" event={"ID":"77ec7474-5075-48b6-8fba-b37947ad9c5a","Type":"ContainerStarted","Data":"1854f99cbec8044ef5f2faf499d5d9055449c831b1ee801047bfee131e715cd1"}
May 25 16:36:37.325509 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.325355    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal" event={"ID":"6cfdc7d2df72a17445303fc2d13b0a7e","Type":"ContainerStarted","Data":"b66653f581cd3d0899597e88e4fa371fd6556e9b1a8c70635a876f8ca84229df"}
May 25 16:36:37.353007 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.352918    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bx8kp" event={"ID":"645b9dd9-5a8a-406b-a2af-5832530e4664","Type":"ContainerStarted","Data":"da8c1ed835168fd8280a32bb7d40071904ab163e54703507538330777e663e2c"}
May 25 16:36:37.367803 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.367738    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n7gn7" event={"ID":"6cc91204-a4f1-4108-b4d6-dab40dbda044","Type":"ContainerStarted","Data":"427908d24746ff9ce9b13a288357872745821c057ee7ce94a4b22808f94bb16b"}
May 25 16:36:37.381133 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.381097    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vx8bb" event={"ID":"d4f109bb-cd5d-47ea-a8d1-af800dc99307","Type":"ContainerStarted","Data":"df3feffbb6041f2392d3be8ef2d594b64eba649559de81d34285b16ee9b067e4"}
May 25 16:36:37.391950 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.391912    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wp4m6" event={"ID":"d15b78cb-69ab-4017-8bae-92669f025b54","Type":"ContainerStarted","Data":"6b92e72814c913836f1b2008e8c49b2015807dbd2fbb1993e391b953103acfaf"}
May 25 16:36:37.410282 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.410245    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nbfsz" event={"ID":"9e86de1e-2942-4e72-b2e7-5243b02ed554","Type":"ContainerStarted","Data":"0e69bb84ed9e155387e73d1cb3f71aee133a68ed41d67756813e1017d73cdd7b"}
May 25 16:36:37.427755 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.427717    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xfhkv" event={"ID":"1018b970-9e3e-4eb4-a045-17340c9ea537","Type":"ContainerStarted","Data":"91bb239689b091692e46b47a2380c63ebb827a63bfebe6386be7aeb7c7c51893"}
May 25 16:36:37.449338 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.449287    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg" event={"ID":"230e7d42-ab66-49b4-876c-969beac2ebca","Type":"ContainerStarted","Data":"046f0ba89bfe0e842787f82ede405522ff5415ab9f3234d14b563c1575bfcd8c"}
May 25 16:36:37.774684 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.774133    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret\") pod \"global-pull-secret-syncer-bdqw9\" (UID: \"eb80c465-3f53-4d73-abef-d8465200fa2c\") " pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:37.774684 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:37.774272    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 25 16:36:37.774684 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:37.774335    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret podName:eb80c465-3f53-4d73-abef-d8465200fa2c nodeName:}" failed. No retries permitted until 2026-05-25 16:36:39.774317267 +0000 UTC m=+5.122235505 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret") pod "global-pull-secret-syncer-bdqw9" (UID: "eb80c465-3f53-4d73-abef-d8465200fa2c") : object "kube-system"/"original-pull-secret" not registered
May 25 16:36:37.875258 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.875203    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g7tfc\" (UniqueName: \"kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc\") pod \"network-check-target-fxfkl\" (UID: \"11062d63-8afe-498a-8186-ba3987ab73d4\") " pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:37.875453 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:37.875295    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs\") pod \"network-metrics-daemon-l57q2\" (UID: \"6646a861-65d0-4b6f-af0c-415ff46851dd\") " pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:37.875453 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:37.875403    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 16:36:37.875453 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:37.875433    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 16:36:37.875453 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:37.875449    2554 projected.go:194] Error preparing data for projected volume kube-api-access-g7tfc for pod openshift-network-diagnostics/network-check-target-fxfkl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 16:36:37.875656 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:37.875408    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 16:36:37.875656 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:37.875511    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc podName:11062d63-8afe-498a-8186-ba3987ab73d4 nodeName:}" failed. No retries permitted until 2026-05-25 16:36:39.875491696 +0000 UTC m=+5.223409929 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-g7tfc" (UniqueName: "kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc") pod "network-check-target-fxfkl" (UID: "11062d63-8afe-498a-8186-ba3987ab73d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 16:36:37.875656 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:37.875558    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs podName:6646a861-65d0-4b6f-af0c-415ff46851dd nodeName:}" failed. No retries permitted until 2026-05-25 16:36:39.87553779 +0000 UTC m=+5.223456023 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs") pod "network-metrics-daemon-l57q2" (UID: "6646a861-65d0-4b6f-af0c-415ff46851dd") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 16:36:38.262463 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:38.262370    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:38.262651 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:38.262498    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:36:38.262940 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:38.262919    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:38.263055 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:38.263016    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:36:38.272919 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:38.272860    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-24 16:31:36 +0000 UTC" deadline="2028-02-19 14:58:09.908824292 +0000 UTC"
May 25 16:36:38.272919 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:38.272909    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15238h21m31.635919052s"
May 25 16:36:38.430985 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:38.430764    2554 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 16:36:39.262562 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:39.262470    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:39.262738 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:39.262682    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:36:39.791150 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:39.791043    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret\") pod \"global-pull-secret-syncer-bdqw9\" (UID: \"eb80c465-3f53-4d73-abef-d8465200fa2c\") " pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:39.791621 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:39.791209    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 25 16:36:39.791621 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:39.791284    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret podName:eb80c465-3f53-4d73-abef-d8465200fa2c nodeName:}" failed. No retries permitted until 2026-05-25 16:36:43.79126388 +0000 UTC m=+9.139182115 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret") pod "global-pull-secret-syncer-bdqw9" (UID: "eb80c465-3f53-4d73-abef-d8465200fa2c") : object "kube-system"/"original-pull-secret" not registered
May 25 16:36:39.891915 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:39.891861    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g7tfc\" (UniqueName: \"kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc\") pod \"network-check-target-fxfkl\" (UID: \"11062d63-8afe-498a-8186-ba3987ab73d4\") " pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:39.892102 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:39.891974    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs\") pod \"network-metrics-daemon-l57q2\" (UID: \"6646a861-65d0-4b6f-af0c-415ff46851dd\") " pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:39.892102 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:39.892095    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 16:36:39.892235 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:39.892120    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 16:36:39.892235 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:39.892145    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 16:36:39.892235 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:39.892159    2554 projected.go:194] Error preparing data for projected volume kube-api-access-g7tfc for pod openshift-network-diagnostics/network-check-target-fxfkl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 16:36:39.892235 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:39.892161    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs podName:6646a861-65d0-4b6f-af0c-415ff46851dd nodeName:}" failed. No retries permitted until 2026-05-25 16:36:43.892140922 +0000 UTC m=+9.240059154 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs") pod "network-metrics-daemon-l57q2" (UID: "6646a861-65d0-4b6f-af0c-415ff46851dd") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 16:36:39.892235 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:39.892222    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc podName:11062d63-8afe-498a-8186-ba3987ab73d4 nodeName:}" failed. No retries permitted until 2026-05-25 16:36:43.892206374 +0000 UTC m=+9.240124608 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-g7tfc" (UniqueName: "kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc") pod "network-check-target-fxfkl" (UID: "11062d63-8afe-498a-8186-ba3987ab73d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 16:36:40.262641 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:40.262607    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:40.262641 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:40.262634    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:40.262903 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:40.262745    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:36:40.262903 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:40.262805    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:36:41.262343 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:41.262311    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:41.262766 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:41.262451    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:36:42.262077 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:42.261733    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:42.262077 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:42.261776    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:42.262077 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:42.261900    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:36:42.262374 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:42.262342    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:36:43.262004 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:43.261966    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:43.262187 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:43.262136    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:36:43.824132 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:43.824009    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret\") pod \"global-pull-secret-syncer-bdqw9\" (UID: \"eb80c465-3f53-4d73-abef-d8465200fa2c\") " pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:43.824613 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:43.824140    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 25 16:36:43.824613 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:43.824231    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret podName:eb80c465-3f53-4d73-abef-d8465200fa2c nodeName:}" failed. No retries permitted until 2026-05-25 16:36:51.824188077 +0000 UTC m=+17.172106306 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret") pod "global-pull-secret-syncer-bdqw9" (UID: "eb80c465-3f53-4d73-abef-d8465200fa2c") : object "kube-system"/"original-pull-secret" not registered
May 25 16:36:43.925221 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:43.924889    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs\") pod \"network-metrics-daemon-l57q2\" (UID: \"6646a861-65d0-4b6f-af0c-415ff46851dd\") " pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:43.925221 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:43.924962    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g7tfc\" (UniqueName: \"kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc\") pod \"network-check-target-fxfkl\" (UID: \"11062d63-8afe-498a-8186-ba3987ab73d4\") " pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:43.925221 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:43.925031    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 16:36:43.925221 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:43.925101    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 16:36:43.925221 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:43.925115    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs podName:6646a861-65d0-4b6f-af0c-415ff46851dd nodeName:}" failed. No retries permitted until 2026-05-25 16:36:51.925092182 +0000 UTC m=+17.273010423 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs") pod "network-metrics-daemon-l57q2" (UID: "6646a861-65d0-4b6f-af0c-415ff46851dd") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 16:36:43.925221 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:43.925118    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 16:36:43.925221 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:43.925134    2554 projected.go:194] Error preparing data for projected volume kube-api-access-g7tfc for pod openshift-network-diagnostics/network-check-target-fxfkl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 16:36:43.925221 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:43.925186    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc podName:11062d63-8afe-498a-8186-ba3987ab73d4 nodeName:}" failed. No retries permitted until 2026-05-25 16:36:51.925171203 +0000 UTC m=+17.273089439 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-g7tfc" (UniqueName: "kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc") pod "network-check-target-fxfkl" (UID: "11062d63-8afe-498a-8186-ba3987ab73d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 16:36:44.262742 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:44.262700    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:44.262742 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:44.262748    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:44.263008 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:44.262834    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:36:44.263063 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:44.263003    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:36:45.263713 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:45.263537    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:45.263713 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:45.263674    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:36:46.262111 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:46.262073    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:46.262305 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:46.262073    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:46.262305 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:46.262214    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:36:46.262414 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:46.262324    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:36:47.262727 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:47.262695    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:47.263167 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:47.262820    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:36:48.262557 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:48.262522    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:48.262734 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:48.262527    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:48.262734 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:48.262643    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:36:48.262734 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:48.262717    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:36:49.262027 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:49.261988    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:49.262209 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:49.262123    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:36:50.262012 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:50.261966    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:50.262012 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:50.262007    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:50.262486 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:50.262093    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:36:50.262486 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:50.262240    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:36:51.262326 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:51.262295    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:51.262708 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:51.262426    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:36:51.883106 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:51.883065    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret\") pod \"global-pull-secret-syncer-bdqw9\" (UID: \"eb80c465-3f53-4d73-abef-d8465200fa2c\") " pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:51.883358 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:51.883226    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 25 16:36:51.883358 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:51.883312    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret podName:eb80c465-3f53-4d73-abef-d8465200fa2c nodeName:}" failed. No retries permitted until 2026-05-25 16:37:07.88328983 +0000 UTC m=+33.231208077 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret") pod "global-pull-secret-syncer-bdqw9" (UID: "eb80c465-3f53-4d73-abef-d8465200fa2c") : object "kube-system"/"original-pull-secret" not registered
May 25 16:36:51.984176 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:51.984127    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g7tfc\" (UniqueName: \"kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc\") pod \"network-check-target-fxfkl\" (UID: \"11062d63-8afe-498a-8186-ba3987ab73d4\") " pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:51.984348 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:51.984223    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs\") pod \"network-metrics-daemon-l57q2\" (UID: \"6646a861-65d0-4b6f-af0c-415ff46851dd\") " pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:51.984348 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:51.984309    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 16:36:51.984348 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:51.984332    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 16:36:51.984348 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:51.984333    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 16:36:51.984348 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:51.984342    2554 projected.go:194] Error preparing data for projected volume kube-api-access-g7tfc for pod openshift-network-diagnostics/network-check-target-fxfkl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 16:36:51.984538 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:51.984405    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc podName:11062d63-8afe-498a-8186-ba3987ab73d4 nodeName:}" failed. No retries permitted until 2026-05-25 16:37:07.98438313 +0000 UTC m=+33.332301359 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-g7tfc" (UniqueName: "kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc") pod "network-check-target-fxfkl" (UID: "11062d63-8afe-498a-8186-ba3987ab73d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 16:36:51.984538 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:51.984429    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs podName:6646a861-65d0-4b6f-af0c-415ff46851dd nodeName:}" failed. No retries permitted until 2026-05-25 16:37:07.984418715 +0000 UTC m=+33.332336946 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs") pod "network-metrics-daemon-l57q2" (UID: "6646a861-65d0-4b6f-af0c-415ff46851dd") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 16:36:52.265262 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:52.262997    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:52.265262 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:52.263175    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:36:52.265262 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:52.263009    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:52.265262 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:52.263704    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:36:53.262634 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:53.262597    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:53.262804 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:53.262735    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:36:54.262009 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:54.261974    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:54.262009 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:54.261999    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:54.262465 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:54.262099    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:36:54.262465 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:54.262240    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:36:55.263505 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:55.263468    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:55.263940 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:55.263602    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:36:56.262778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:56.262544    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:56.262939 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:56.262544    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:56.263006 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:56.262969    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:36:56.263006 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:56.262986    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:36:56.496401 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:56.495887    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vx8bb" event={"ID":"d4f109bb-cd5d-47ea-a8d1-af800dc99307","Type":"ContainerStarted","Data":"ddc64ec7df34e98fa5ac265b8a7d5a7e03031f429cf4a372b035ad5bb143e048"}
May 25 16:36:56.498149 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:56.498114    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nbfsz" event={"ID":"9e86de1e-2942-4e72-b2e7-5243b02ed554","Type":"ContainerStarted","Data":"23b280def4fe72f25da29d73f23e409c9a386e6b015c07d6bbe550e8f81c2d03"}
May 25 16:36:56.510793 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:56.510680    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 16:36:56.511106 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:56.511077    2554 generic.go:358] "Generic (PLEG): container finished" podID="230e7d42-ab66-49b4-876c-969beac2ebca" containerID="adb4329a37db1f99ce08c1ddb38bd42442b40e08da62824312f05094df653ac1" exitCode=1
May 25 16:36:56.511212 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:56.511163    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg" event={"ID":"230e7d42-ab66-49b4-876c-969beac2ebca","Type":"ContainerStarted","Data":"1e8f463265c4bb75a3d1806e854a7dcf08e974b37cbd33e55b5c09efbd0ed2f9"}
May 25 16:36:56.511212 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:56.511193    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg" event={"ID":"230e7d42-ab66-49b4-876c-969beac2ebca","Type":"ContainerStarted","Data":"9ce072acaf6edd32fb3b48f1ff2b8133ac865ba117840fbf60f74e5134bde005"}
May 25 16:36:56.511212 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:56.511207    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg" event={"ID":"230e7d42-ab66-49b4-876c-969beac2ebca","Type":"ContainerStarted","Data":"de682db7e192dedb43d371a1f27cb0cbac7724de4aa0f6cf799cebfe4472a677"}
May 25 16:36:56.511353 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:56.511220    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg" event={"ID":"230e7d42-ab66-49b4-876c-969beac2ebca","Type":"ContainerDied","Data":"adb4329a37db1f99ce08c1ddb38bd42442b40e08da62824312f05094df653ac1"}
May 25 16:36:56.511353 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:56.511230    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg" event={"ID":"230e7d42-ab66-49b4-876c-969beac2ebca","Type":"ContainerStarted","Data":"d2e73ad9ef00ca0f43989d8a1cdf5ed3ca85081939a4eac2a25901a152b0a4e3"}
May 25 16:36:56.518281 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:56.518251    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-137.ec2.internal" event={"ID":"9e7d79ffc7bf90f02369e701a94a0e0a","Type":"ContainerStarted","Data":"d63b68b5e6e7862a892115acdd6ce11c17256c34b21dfdbf8e649fefc1d67e7f"}
May 25 16:36:56.528526 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:56.528473    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vx8bb" podStartSLOduration=2.270170104 podStartE2EDuration="21.52845382s" podCreationTimestamp="2026-05-25 16:36:35 +0000 UTC" firstStartedPulling="2026-05-25 16:36:36.585981029 +0000 UTC m=+1.933899258" lastFinishedPulling="2026-05-25 16:36:55.844264741 +0000 UTC m=+21.192182974" observedRunningTime="2026-05-25 16:36:56.513829022 +0000 UTC m=+21.861747274" watchObservedRunningTime="2026-05-25 16:36:56.52845382 +0000 UTC m=+21.876372069"
May 25 16:36:56.528839 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:56.528815    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-nbfsz" podStartSLOduration=1.801763652 podStartE2EDuration="21.528806981s" podCreationTimestamp="2026-05-25 16:36:35 +0000 UTC" firstStartedPulling="2026-05-25 16:36:36.539910553 +0000 UTC m=+1.887828782" lastFinishedPulling="2026-05-25 16:36:56.266953874 +0000 UTC m=+21.614872111" observedRunningTime="2026-05-25 16:36:56.528423635 +0000 UTC m=+21.876341887" watchObservedRunningTime="2026-05-25 16:36:56.528806981 +0000 UTC m=+21.876725232"
May 25 16:36:56.549632 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:56.547788    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-137.ec2.internal" podStartSLOduration=20.547769203 podStartE2EDuration="20.547769203s" podCreationTimestamp="2026-05-25 16:36:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 16:36:56.547548601 +0000 UTC m=+21.895467063" watchObservedRunningTime="2026-05-25 16:36:56.547769203 +0000 UTC m=+21.895687460"
May 25 16:36:57.262503 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.262463    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:57.262719 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:57.262613    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:36:57.513159 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.513137    2554 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 25 16:36:57.523025 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.523000    2554 generic.go:358] "Generic (PLEG): container finished" podID="6cc91204-a4f1-4108-b4d6-dab40dbda044" containerID="52cef4aa6b9a2150ce7b40eaf9480d350c635afa811d400578f4aab1fb8821ec" exitCode=0
May 25 16:36:57.523165 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.523074    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n7gn7" event={"ID":"6cc91204-a4f1-4108-b4d6-dab40dbda044","Type":"ContainerDied","Data":"52cef4aa6b9a2150ce7b40eaf9480d350c635afa811d400578f4aab1fb8821ec"}
May 25 16:36:57.524302 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.524272    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wp4m6" event={"ID":"d15b78cb-69ab-4017-8bae-92669f025b54","Type":"ContainerStarted","Data":"b6868591dc47313d87d23c55364431194c43c82283617f45a8c267dc3324e44f"}
May 25 16:36:57.525574 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.525541    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xfhkv" event={"ID":"1018b970-9e3e-4eb4-a045-17340c9ea537","Type":"ContainerStarted","Data":"8df24e3e6f74603715a2a5d4a90e34de649cc3f1dad454db2d8dd2bd91baca04"}
May 25 16:36:57.527892 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.527859    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 16:36:57.528198 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.528175    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg" event={"ID":"230e7d42-ab66-49b4-876c-969beac2ebca","Type":"ContainerStarted","Data":"bc55a2497a940f7c01a12997e3b6f1026a3972c265cf87485c3e53fb5d64cf81"}
May 25 16:36:57.529421 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.529391    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7klm9" event={"ID":"f603e2fb-4fa8-4746-a721-fb1e1342fc6b","Type":"ContainerStarted","Data":"96fb331477f8809349268828bf80efd791099e01397e02ccd554825a37a246ae"}
May 25 16:36:57.530839 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.530819    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888" event={"ID":"77ec7474-5075-48b6-8fba-b37947ad9c5a","Type":"ContainerStarted","Data":"90910145ae42cc9825d5830af9d498c34ab1a40c779b7d1e3168563412875bb5"}
May 25 16:36:57.530933 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.530846    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888" event={"ID":"77ec7474-5075-48b6-8fba-b37947ad9c5a","Type":"ContainerStarted","Data":"6677dad830ddfa9794f72d287ccc198173e09f70a698ee2d9a054374dc8b59aa"}
May 25 16:36:57.532061 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.532040    2554 generic.go:358] "Generic (PLEG): container finished" podID="6cfdc7d2df72a17445303fc2d13b0a7e" containerID="44a5215a67d4127f581ae0b8aa340e00692fbe237fef2a05d93b303c1720a0ef" exitCode=0
May 25 16:36:57.532150 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.532097    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal" event={"ID":"6cfdc7d2df72a17445303fc2d13b0a7e","Type":"ContainerDied","Data":"44a5215a67d4127f581ae0b8aa340e00692fbe237fef2a05d93b303c1720a0ef"}
May 25 16:36:57.533359 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.533335    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bx8kp" event={"ID":"645b9dd9-5a8a-406b-a2af-5832530e4664","Type":"ContainerStarted","Data":"36a18ce28f46125388916ed9b160501144fc388f2320b336f8ddca0ee5c8034c"}
May 25 16:36:57.575985 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.575924    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xfhkv" podStartSLOduration=3.276469389 podStartE2EDuration="22.575906513s" podCreationTimestamp="2026-05-25 16:36:35 +0000 UTC" firstStartedPulling="2026-05-25 16:36:36.518012475 +0000 UTC m=+1.865930705" lastFinishedPulling="2026-05-25 16:36:55.817449601 +0000 UTC m=+21.165367829" observedRunningTime="2026-05-25 16:36:57.575355859 +0000 UTC m=+22.923274111" watchObservedRunningTime="2026-05-25 16:36:57.575906513 +0000 UTC m=+22.923824775"
May 25 16:36:57.607677 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.607627    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wp4m6" podStartSLOduration=3.343361423 podStartE2EDuration="22.607607044s" podCreationTimestamp="2026-05-25 16:36:35 +0000 UTC" firstStartedPulling="2026-05-25 16:36:36.553434585 +0000 UTC m=+1.901352814" lastFinishedPulling="2026-05-25 16:36:55.817680187 +0000 UTC m=+21.165598435" observedRunningTime="2026-05-25 16:36:57.607476793 +0000 UTC m=+22.955395057" watchObservedRunningTime="2026-05-25 16:36:57.607607044 +0000 UTC m=+22.955525297"
May 25 16:36:57.641286 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:57.639201    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7klm9" podStartSLOduration=3.385943461 podStartE2EDuration="22.639181725s" podCreationTimestamp="2026-05-25 16:36:35 +0000 UTC" firstStartedPulling="2026-05-25 16:36:36.590803079 +0000 UTC m=+1.938721308" lastFinishedPulling="2026-05-25 16:36:55.844041329 +0000 UTC m=+21.191959572" observedRunningTime="2026-05-25 16:36:57.621678393 +0000 UTC m=+22.969596656" watchObservedRunningTime="2026-05-25 16:36:57.639181725 +0000 UTC m=+22.987099976"
May 25 16:36:58.225849 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:58.225747    2554 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-25T16:36:57.513155296Z","UUID":"95a965d6-f9e0-48fa-a9cf-f75cdf9be704","Handler":null,"Name":"","Endpoint":""}
May 25 16:36:58.229025 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:58.228999    2554 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 25 16:36:58.229025 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:58.229033    2554 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 25 16:36:58.262528 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:58.262497    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:36:58.262715 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:58.262497    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:36:58.262715 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:58.262629    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:36:58.262829 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:58.262742    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:36:58.537919 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:58.537887    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888" event={"ID":"77ec7474-5075-48b6-8fba-b37947ad9c5a","Type":"ContainerStarted","Data":"e87e5d7690103ad3fa4aad36d68e6337ae4845c98e47a174a45cacad295e5896"}
May 25 16:36:58.540181 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:58.540145    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal" event={"ID":"6cfdc7d2df72a17445303fc2d13b0a7e","Type":"ContainerStarted","Data":"44643edc07a2751e6f5dad8f54275c542462ca4c29d02510b600c962707ddb0b"}
May 25 16:36:58.555774 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:58.555718    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-bx8kp" podStartSLOduration=4.340863517 podStartE2EDuration="23.555699826s" podCreationTimestamp="2026-05-25 16:36:35 +0000 UTC" firstStartedPulling="2026-05-25 16:36:36.602653606 +0000 UTC m=+1.950571849" lastFinishedPulling="2026-05-25 16:36:55.817489923 +0000 UTC m=+21.165408158" observedRunningTime="2026-05-25 16:36:57.640947399 +0000 UTC m=+22.988865630" watchObservedRunningTime="2026-05-25 16:36:58.555699826 +0000 UTC m=+23.903618078"
May 25 16:36:58.577223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:58.577163    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76888" podStartSLOduration=1.783527221 podStartE2EDuration="23.577144722s" podCreationTimestamp="2026-05-25 16:36:35 +0000 UTC" firstStartedPulling="2026-05-25 16:36:36.486435697 +0000 UTC m=+1.834353925" lastFinishedPulling="2026-05-25 16:36:58.280053198 +0000 UTC m=+23.627971426" observedRunningTime="2026-05-25 16:36:58.555443659 +0000 UTC m=+23.903361912" watchObservedRunningTime="2026-05-25 16:36:58.577144722 +0000 UTC m=+23.925062975"
May 25 16:36:59.262083 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:59.262048    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:36:59.262272 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:36:59.262180    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:36:59.545354 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:59.545280    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 16:36:59.545992 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:36:59.545674    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg" event={"ID":"230e7d42-ab66-49b4-876c-969beac2ebca","Type":"ContainerStarted","Data":"adb95f17392210a73e506f7bd7ff62f73ae8700d58ff377c66193b0a1f86aefc"}
May 25 16:37:00.262705 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:00.262664    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:37:00.262931 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:00.262679    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:37:00.262931 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:00.262794    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:37:00.262931 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:00.262844    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:37:00.440947 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:00.440907    2554 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xfhkv"
May 25 16:37:00.441684 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:00.441666    2554 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xfhkv"
May 25 16:37:00.461186 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:00.460214    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-137.ec2.internal" podStartSLOduration=24.460194596 podStartE2EDuration="24.460194596s" podCreationTimestamp="2026-05-25 16:36:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 16:36:58.576659138 +0000 UTC m=+23.924577395" watchObservedRunningTime="2026-05-25 16:37:00.460194596 +0000 UTC m=+25.808112847"
May 25 16:37:00.547614 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:00.547542    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xfhkv"
May 25 16:37:00.548157 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:00.548137    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xfhkv"
May 25 16:37:01.262930 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:01.262652    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:37:01.263216 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:01.263062    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:37:01.553123 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:01.553048    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 16:37:01.553627 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:01.553485    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg" event={"ID":"230e7d42-ab66-49b4-876c-969beac2ebca","Type":"ContainerStarted","Data":"3dc5efddba2f2bb61dbe2725e61db7ed8bb5ef4f3a27236a7c9b93e5b8b333bc"}
May 25 16:37:01.554065 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:01.554044    2554 scope.go:117] "RemoveContainer" containerID="adb4329a37db1f99ce08c1ddb38bd42442b40e08da62824312f05094df653ac1"
May 25 16:37:02.261789 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:02.261748    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:37:02.261789 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:02.261793    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:37:02.262051 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:02.261910    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:37:02.262116 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:02.262065    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:37:02.558232 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:02.558062    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 16:37:02.558618 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:02.558566    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg" event={"ID":"230e7d42-ab66-49b4-876c-969beac2ebca","Type":"ContainerStarted","Data":"3a96e12fe71c596d5687c354830a3df1b3249a252acf82667221cbb2f8e88a45"}
May 25 16:37:02.558937 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:02.558913    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:37:02.558937 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:02.558940    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:37:02.559132 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:02.558951    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:37:02.576984 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:02.576958    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:37:02.577454 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:02.577433    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:37:02.589390 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:02.589344    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg" podStartSLOduration=8.205416878 podStartE2EDuration="27.589330299s" podCreationTimestamp="2026-05-25 16:36:35 +0000 UTC" firstStartedPulling="2026-05-25 16:36:36.49556647 +0000 UTC m=+1.843484700" lastFinishedPulling="2026-05-25 16:36:55.879479888 +0000 UTC m=+21.227398121" observedRunningTime="2026-05-25 16:37:02.587694424 +0000 UTC m=+27.935612689" watchObservedRunningTime="2026-05-25 16:37:02.589330299 +0000 UTC m=+27.937248551"
May 25 16:37:02.658772 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:02.658734    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bdqw9"]
May 25 16:37:02.658940 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:02.658912    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:37:02.659064 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:02.659038    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:37:02.661903 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:02.661857    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l57q2"]
May 25 16:37:02.662055 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:02.662009    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:37:02.662131 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:02.662107    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:37:02.662529 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:02.662510    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fxfkl"]
May 25 16:37:02.662640 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:02.662612    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:37:02.662764 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:02.662745    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:37:03.564543 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:03.564513    2554 generic.go:358] "Generic (PLEG): container finished" podID="6cc91204-a4f1-4108-b4d6-dab40dbda044" containerID="00445754918d3781a0cde9be605820459787b884b1abd95499e8c3d6aaf2359e" exitCode=0
May 25 16:37:03.565016 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:03.564604    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n7gn7" event={"ID":"6cc91204-a4f1-4108-b4d6-dab40dbda044","Type":"ContainerDied","Data":"00445754918d3781a0cde9be605820459787b884b1abd95499e8c3d6aaf2359e"}
May 25 16:37:04.262336 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:04.262299    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:37:04.262532 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:04.262319    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:37:04.262532 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:04.262438    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:37:04.262532 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:04.262447    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:37:04.262686 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:04.262529    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:37:04.262686 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:04.262604    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:37:05.569287 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:05.569185    2554 generic.go:358] "Generic (PLEG): container finished" podID="6cc91204-a4f1-4108-b4d6-dab40dbda044" containerID="3a206522de5ba478ab7a8d7b26bf875e8420b7ca9b38b949cad3742f20154426" exitCode=0
May 25 16:37:05.569287 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:05.569237    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n7gn7" event={"ID":"6cc91204-a4f1-4108-b4d6-dab40dbda044","Type":"ContainerDied","Data":"3a206522de5ba478ab7a8d7b26bf875e8420b7ca9b38b949cad3742f20154426"}
May 25 16:37:06.262894 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:06.262691    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:37:06.263042 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:06.262696    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:37:06.263042 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:06.262996    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:37:06.263186 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:06.262706    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:37:06.263186 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:06.263052    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:37:06.263186 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:06.263139    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:37:06.572776 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:06.572736    2554 generic.go:358] "Generic (PLEG): container finished" podID="6cc91204-a4f1-4108-b4d6-dab40dbda044" containerID="a1a533f06bd4f374b31bec1a8fe42f8ee0ada2e7d02f9d3fd8e1352dbdeaa452" exitCode=0
May 25 16:37:06.573162 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:06.572797    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n7gn7" event={"ID":"6cc91204-a4f1-4108-b4d6-dab40dbda044","Type":"ContainerDied","Data":"a1a533f06bd4f374b31bec1a8fe42f8ee0ada2e7d02f9d3fd8e1352dbdeaa452"}
May 25 16:37:07.905380 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:07.905333    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret\") pod \"global-pull-secret-syncer-bdqw9\" (UID: \"eb80c465-3f53-4d73-abef-d8465200fa2c\") " pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:37:07.905839 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:07.905531    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 25 16:37:07.905839 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:07.905607    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret podName:eb80c465-3f53-4d73-abef-d8465200fa2c nodeName:}" failed. No retries permitted until 2026-05-25 16:37:39.905587253 +0000 UTC m=+65.253505497 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret") pod "global-pull-secret-syncer-bdqw9" (UID: "eb80c465-3f53-4d73-abef-d8465200fa2c") : object "kube-system"/"original-pull-secret" not registered
May 25 16:37:08.006762 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.006724    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g7tfc\" (UniqueName: \"kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc\") pod \"network-check-target-fxfkl\" (UID: \"11062d63-8afe-498a-8186-ba3987ab73d4\") " pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:37:08.006946 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.006806    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs\") pod \"network-metrics-daemon-l57q2\" (UID: \"6646a861-65d0-4b6f-af0c-415ff46851dd\") " pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:37:08.006946 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:08.006908    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 16:37:08.006946 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:08.006934    2554 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 16:37:08.006946 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:08.006947    2554 projected.go:194] Error preparing data for projected volume kube-api-access-g7tfc for pod openshift-network-diagnostics/network-check-target-fxfkl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 16:37:08.007085 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:08.006917    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 16:37:08.007085 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:08.007004    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc podName:11062d63-8afe-498a-8186-ba3987ab73d4 nodeName:}" failed. No retries permitted until 2026-05-25 16:37:40.006984203 +0000 UTC m=+65.354902434 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-g7tfc" (UniqueName: "kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc") pod "network-check-target-fxfkl" (UID: "11062d63-8afe-498a-8186-ba3987ab73d4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 16:37:08.007085 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:08.007030    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs podName:6646a861-65d0-4b6f-af0c-415ff46851dd nodeName:}" failed. No retries permitted until 2026-05-25 16:37:40.007017196 +0000 UTC m=+65.354935425 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs") pod "network-metrics-daemon-l57q2" (UID: "6646a861-65d0-4b6f-af0c-415ff46851dd") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 16:37:08.262627 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.262594    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:37:08.262825 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.262594    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:37:08.262825 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:08.262722    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxfkl" podUID="11062d63-8afe-498a-8186-ba3987ab73d4"
May 25 16:37:08.262825 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.262594    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:37:08.262825 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:08.262817    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:37:08.263068 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:08.262925    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdqw9" podUID="eb80c465-3f53-4d73-abef-d8465200fa2c"
May 25 16:37:08.915307 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.915229    2554 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-137.ec2.internal" event="NodeReady"
May 25 16:37:08.915731 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.915375    2554 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 25 16:37:08.961296 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.960813    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"]
May 25 16:37:08.965691 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.965668    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74bf8cf786-frqqw"]
May 25 16:37:08.965894 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.965859    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:37:08.968892 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.968846    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
May 25 16:37:08.969023 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.968908    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
May 25 16:37:08.969151 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.969129    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-f46lz\""
May 25 16:37:08.969464 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.969445    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"]
May 25 16:37:08.969614 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.969599    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:08.973542 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.973407    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:08.973695 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.973598    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 25 16:37:08.973695 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.973678    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 25 16:37:08.973909 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.973845    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 25 16:37:08.974252 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.974223    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6lmcz\""
May 25 16:37:08.974409 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.974395    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"]
May 25 16:37:08.978146 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.978085    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b"]
May 25 16:37:08.978409 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.978360    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"
May 25 16:37:08.980364 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.980081    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 25 16:37:08.982077 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.981060    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
May 25 16:37:08.982077 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.981545    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
May 25 16:37:08.982077 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.981704    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
May 25 16:37:08.982272 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.982204    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
May 25 16:37:08.982657 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.982331    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
May 25 16:37:08.982657 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.982373    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
May 25 16:37:08.982657 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.982413    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
May 25 16:37:08.986188 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.986155    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
May 25 16:37:08.986304 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.986277    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"]
May 25 16:37:08.986406 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.986381    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b"
May 25 16:37:08.989417 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.989374    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74bf8cf786-frqqw"]
May 25 16:37:08.990055 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.990025    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-x5c4h\""
May 25 16:37:08.990222 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.990179    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
May 25 16:37:08.998698 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.998673    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"]
May 25 16:37:08.999620 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:08.999570    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lndmn"]
May 25 16:37:09.005415 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.005391    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-l2p4k"]
May 25 16:37:09.005576 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.005558    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lndmn"
May 25 16:37:09.008511 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.008489    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 25 16:37:09.008884 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.008851    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"]
May 25 16:37:09.008979 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.008919    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b"]
May 25 16:37:09.009089 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.009070    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:37:09.009237 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.009219    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rqkd2\""
May 25 16:37:09.009343 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.009328    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 25 16:37:09.011175 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.011155    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l2p4k"]
May 25 16:37:09.011853 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.011830    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 25 16:37:09.011967 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.011946    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 25 16:37:09.012114 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.012097    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mbvc2\""
May 25 16:37:09.012687 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.012667    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 25 16:37:09.019723 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.019699    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lndmn"]
May 25 16:37:09.113568 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.113525    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/882706e9-6a8b-4f16-8106-f30d26775096-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.113738 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.113586    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.113738 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.113621    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:09.113738 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.113640    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ba6020a-e04d-4c62-836f-058b291b1d4b-trusted-ca\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.113738 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.113664    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzzzm\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-kube-api-access-hzzzm\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.113738 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.113714    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dba85d0a-072c-4a30-b169-92843a576b3b-tmp-dir\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:09.113986 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.113742    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/65ea9255-03e8-4801-b096-3e2b106c5bd1-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d678c9b9b-dbqhl\" (UID: \"65ea9255-03e8-4801-b096-3e2b106c5bd1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"
May 25 16:37:09.113986 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.113772    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14a16fa6-ae30-45ac-a835-311d9a99e2c3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-d54f7fc8-dns4b\" (UID: \"14a16fa6-ae30-45ac-a835-311d9a99e2c3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b"
May 25 16:37:09.113986 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.113803    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-bound-sa-token\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.113986 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.113831    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9nmf\" (UniqueName: \"kubernetes.io/projected/7a41c3db-9f6d-4e76-9b35-839fa3b35665-kube-api-access-s9nmf\") pod \"ingress-canary-l2p4k\" (UID: \"7a41c3db-9f6d-4e76-9b35-839fa3b35665\") " pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:37:09.113986 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.113856    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/882706e9-6a8b-4f16-8106-f30d26775096-ca\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.113986 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.113895    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrdbz\" (UniqueName: \"kubernetes.io/projected/882706e9-6a8b-4f16-8106-f30d26775096-kube-api-access-qrdbz\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.113986 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.113927    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/882706e9-6a8b-4f16-8106-f30d26775096-hub\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.113986 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.113953    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/882706e9-6a8b-4f16-8106-f30d26775096-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.114259 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.113998    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ba6020a-e04d-4c62-836f-058b291b1d4b-installation-pull-secrets\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.114259 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.114086    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxrgt\" (UniqueName: \"kubernetes.io/projected/14a16fa6-ae30-45ac-a835-311d9a99e2c3-kube-api-access-vxrgt\") pod \"managed-serviceaccount-addon-agent-d54f7fc8-dns4b\" (UID: \"14a16fa6-ae30-45ac-a835-311d9a99e2c3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b"
May 25 16:37:09.114259 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.114175    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/02a5243d-f581-40f8-a48a-b2cfd2296e89-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-jsgsj\" (UID: \"02a5243d-f581-40f8-a48a-b2cfd2296e89\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:37:09.114259 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.114201    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert\") pod \"ingress-canary-l2p4k\" (UID: \"7a41c3db-9f6d-4e76-9b35-839fa3b35665\") " pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:37:09.114404 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.114294    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-certificates\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.114404 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.114335    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ba6020a-e04d-4c62-836f-058b291b1d4b-image-registry-private-configuration\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.114404 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.114374    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/882706e9-6a8b-4f16-8106-f30d26775096-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.114404 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.114392    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z92zn\" (UniqueName: \"kubernetes.io/projected/65ea9255-03e8-4801-b096-3e2b106c5bd1-kube-api-access-z92zn\") pod \"klusterlet-addon-workmgr-5d678c9b9b-dbqhl\" (UID: \"65ea9255-03e8-4801-b096-3e2b106c5bd1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"
May 25 16:37:09.114566 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.114413    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lqv6\" (UniqueName: \"kubernetes.io/projected/dba85d0a-072c-4a30-b169-92843a576b3b-kube-api-access-6lqv6\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:09.114566 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.114438    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ba6020a-e04d-4c62-836f-058b291b1d4b-ca-trust-extracted\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.114566 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.114475    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-jsgsj\" (UID: \"02a5243d-f581-40f8-a48a-b2cfd2296e89\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:37:09.114566 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.114500    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dba85d0a-072c-4a30-b169-92843a576b3b-config-volume\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:09.114566 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.114520    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65ea9255-03e8-4801-b096-3e2b106c5bd1-tmp\") pod \"klusterlet-addon-workmgr-5d678c9b9b-dbqhl\" (UID: \"65ea9255-03e8-4801-b096-3e2b106c5bd1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"
May 25 16:37:09.215946 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.215853    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.215946 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.215925    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:09.216164 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.215952    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ba6020a-e04d-4c62-836f-058b291b1d4b-trusted-ca\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.216164 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.215979    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hzzzm\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-kube-api-access-hzzzm\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.216164 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.216009    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 16:37:09.216164 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.216031    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74bf8cf786-frqqw: secret "image-registry-tls" not found
May 25 16:37:09.216164 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.216089    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 16:37:09.216164 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.216092    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls podName:5ba6020a-e04d-4c62-836f-058b291b1d4b nodeName:}" failed. No retries permitted until 2026-05-25 16:37:09.716071757 +0000 UTC m=+35.063989989 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls") pod "image-registry-74bf8cf786-frqqw" (UID: "5ba6020a-e04d-4c62-836f-058b291b1d4b") : secret "image-registry-tls" not found
May 25 16:37:09.216164 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.216156    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls podName:dba85d0a-072c-4a30-b169-92843a576b3b nodeName:}" failed. No retries permitted until 2026-05-25 16:37:09.716137016 +0000 UTC m=+35.064055260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls") pod "dns-default-lndmn" (UID: "dba85d0a-072c-4a30-b169-92843a576b3b") : secret "dns-default-metrics-tls" not found
May 25 16:37:09.216164 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216014    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dba85d0a-072c-4a30-b169-92843a576b3b-tmp-dir\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:09.216539 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216203    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/65ea9255-03e8-4801-b096-3e2b106c5bd1-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d678c9b9b-dbqhl\" (UID: \"65ea9255-03e8-4801-b096-3e2b106c5bd1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"
May 25 16:37:09.216539 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216236    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14a16fa6-ae30-45ac-a835-311d9a99e2c3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-d54f7fc8-dns4b\" (UID: \"14a16fa6-ae30-45ac-a835-311d9a99e2c3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b"
May 25 16:37:09.216539 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216262    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-bound-sa-token\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.216539 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216291    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s9nmf\" (UniqueName: \"kubernetes.io/projected/7a41c3db-9f6d-4e76-9b35-839fa3b35665-kube-api-access-s9nmf\") pod \"ingress-canary-l2p4k\" (UID: \"7a41c3db-9f6d-4e76-9b35-839fa3b35665\") " pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:37:09.216539 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216319    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/882706e9-6a8b-4f16-8106-f30d26775096-ca\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.216539 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216342    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dba85d0a-072c-4a30-b169-92843a576b3b-tmp-dir\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:09.216539 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216345    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qrdbz\" (UniqueName: \"kubernetes.io/projected/882706e9-6a8b-4f16-8106-f30d26775096-kube-api-access-qrdbz\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.216539 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216406    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/882706e9-6a8b-4f16-8106-f30d26775096-hub\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.216539 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216438    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/882706e9-6a8b-4f16-8106-f30d26775096-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.216539 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216463    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ba6020a-e04d-4c62-836f-058b291b1d4b-installation-pull-secrets\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.217081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216599    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vxrgt\" (UniqueName: \"kubernetes.io/projected/14a16fa6-ae30-45ac-a835-311d9a99e2c3-kube-api-access-vxrgt\") pod \"managed-serviceaccount-addon-agent-d54f7fc8-dns4b\" (UID: \"14a16fa6-ae30-45ac-a835-311d9a99e2c3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b"
May 25 16:37:09.217081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216649    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/02a5243d-f581-40f8-a48a-b2cfd2296e89-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-jsgsj\" (UID: \"02a5243d-f581-40f8-a48a-b2cfd2296e89\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:37:09.217081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216675    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert\") pod \"ingress-canary-l2p4k\" (UID: \"7a41c3db-9f6d-4e76-9b35-839fa3b35665\") " pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:37:09.217081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216725    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-certificates\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.217081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216753    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ba6020a-e04d-4c62-836f-058b291b1d4b-image-registry-private-configuration\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.217081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216788    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/882706e9-6a8b-4f16-8106-f30d26775096-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.217081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216815    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z92zn\" (UniqueName: \"kubernetes.io/projected/65ea9255-03e8-4801-b096-3e2b106c5bd1-kube-api-access-z92zn\") pod \"klusterlet-addon-workmgr-5d678c9b9b-dbqhl\" (UID: \"65ea9255-03e8-4801-b096-3e2b106c5bd1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"
May 25 16:37:09.217081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216841    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6lqv6\" (UniqueName: \"kubernetes.io/projected/dba85d0a-072c-4a30-b169-92843a576b3b-kube-api-access-6lqv6\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:09.217081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216888    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ba6020a-e04d-4c62-836f-058b291b1d4b-ca-trust-extracted\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.217081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216922    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-jsgsj\" (UID: \"02a5243d-f581-40f8-a48a-b2cfd2296e89\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:37:09.217081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216948    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dba85d0a-072c-4a30-b169-92843a576b3b-config-volume\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:09.217081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.216974    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65ea9255-03e8-4801-b096-3e2b106c5bd1-tmp\") pod \"klusterlet-addon-workmgr-5d678c9b9b-dbqhl\" (UID: \"65ea9255-03e8-4801-b096-3e2b106c5bd1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"
May 25 16:37:09.217081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.217013    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/882706e9-6a8b-4f16-8106-f30d26775096-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.217659 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.217366    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ba6020a-e04d-4c62-836f-058b291b1d4b-ca-trust-extracted\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.217659 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.217477    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-certificates\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.218408 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.218271    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ba6020a-e04d-4c62-836f-058b291b1d4b-trusted-ca\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.219095 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.219071    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/02a5243d-f581-40f8-a48a-b2cfd2296e89-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-jsgsj\" (UID: \"02a5243d-f581-40f8-a48a-b2cfd2296e89\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:37:09.219200 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.219170    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 16:37:09.219259 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.219236    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert podName:7a41c3db-9f6d-4e76-9b35-839fa3b35665 nodeName:}" failed. No retries permitted until 2026-05-25 16:37:09.719209043 +0000 UTC m=+35.067127272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert") pod "ingress-canary-l2p4k" (UID: "7a41c3db-9f6d-4e76-9b35-839fa3b35665") : secret "canary-serving-cert" not found
May 25 16:37:09.219858 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.219831    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dba85d0a-072c-4a30-b169-92843a576b3b-config-volume\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:09.219961 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.219947    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 25 16:37:09.220022 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.219990    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert podName:02a5243d-f581-40f8-a48a-b2cfd2296e89 nodeName:}" failed. No retries permitted until 2026-05-25 16:37:09.71997658 +0000 UTC m=+35.067894810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-jsgsj" (UID: "02a5243d-f581-40f8-a48a-b2cfd2296e89") : secret "networking-console-plugin-cert" not found
May 25 16:37:09.220757 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.220731    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/882706e9-6a8b-4f16-8106-f30d26775096-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.222157 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.221524    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65ea9255-03e8-4801-b096-3e2b106c5bd1-tmp\") pod \"klusterlet-addon-workmgr-5d678c9b9b-dbqhl\" (UID: \"65ea9255-03e8-4801-b096-3e2b106c5bd1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"
May 25 16:37:09.222157 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.221964    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/882706e9-6a8b-4f16-8106-f30d26775096-ca\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.222157 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.222077    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/882706e9-6a8b-4f16-8106-f30d26775096-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.222157 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.222114    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/882706e9-6a8b-4f16-8106-f30d26775096-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.222396 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.222242    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ba6020a-e04d-4c62-836f-058b291b1d4b-installation-pull-secrets\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.222473 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.222417    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14a16fa6-ae30-45ac-a835-311d9a99e2c3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-d54f7fc8-dns4b\" (UID: \"14a16fa6-ae30-45ac-a835-311d9a99e2c3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b"
May 25 16:37:09.222517 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.222477    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/882706e9-6a8b-4f16-8106-f30d26775096-hub\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.222793 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.222773    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ba6020a-e04d-4c62-836f-058b291b1d4b-image-registry-private-configuration\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.223781 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.223761    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/65ea9255-03e8-4801-b096-3e2b106c5bd1-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d678c9b9b-dbqhl\" (UID: \"65ea9255-03e8-4801-b096-3e2b106c5bd1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"
May 25 16:37:09.226809 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.226657    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9nmf\" (UniqueName: \"kubernetes.io/projected/7a41c3db-9f6d-4e76-9b35-839fa3b35665-kube-api-access-s9nmf\") pod \"ingress-canary-l2p4k\" (UID: \"7a41c3db-9f6d-4e76-9b35-839fa3b35665\") " pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:37:09.226809 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.226717    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxrgt\" (UniqueName: \"kubernetes.io/projected/14a16fa6-ae30-45ac-a835-311d9a99e2c3-kube-api-access-vxrgt\") pod \"managed-serviceaccount-addon-agent-d54f7fc8-dns4b\" (UID: \"14a16fa6-ae30-45ac-a835-311d9a99e2c3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b"
May 25 16:37:09.226809 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.226780    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzzzm\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-kube-api-access-hzzzm\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.227808 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.227782    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrdbz\" (UniqueName: \"kubernetes.io/projected/882706e9-6a8b-4f16-8106-f30d26775096-kube-api-access-qrdbz\") pod \"cluster-proxy-proxy-agent-67b7fccbcc-f5czr\" (UID: \"882706e9-6a8b-4f16-8106-f30d26775096\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.227930 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.227915    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-bound-sa-token\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.229275 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.229251    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lqv6\" (UniqueName: \"kubernetes.io/projected/dba85d0a-072c-4a30-b169-92843a576b3b-kube-api-access-6lqv6\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:09.230244 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.230218    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z92zn\" (UniqueName: \"kubernetes.io/projected/65ea9255-03e8-4801-b096-3e2b106c5bd1-kube-api-access-z92zn\") pod \"klusterlet-addon-workmgr-5d678c9b9b-dbqhl\" (UID: \"65ea9255-03e8-4801-b096-3e2b106c5bd1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"
May 25 16:37:09.309908 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.309853    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:37:09.318133 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.318103    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"
May 25 16:37:09.326880 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.326830    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b"
May 25 16:37:09.501041 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.500950    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"]
May 25 16:37:09.505250 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:37:09.505213    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod882706e9_6a8b_4f16_8106_f30d26775096.slice/crio-a03aa33a23d235c21506f69c92d69719dfef4f484d6c18a2c4c63e3fbfc87559 WatchSource:0}: Error finding container a03aa33a23d235c21506f69c92d69719dfef4f484d6c18a2c4c63e3fbfc87559: Status 404 returned error can't find the container with id a03aa33a23d235c21506f69c92d69719dfef4f484d6c18a2c4c63e3fbfc87559
May 25 16:37:09.508579 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.508421    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b"]
May 25 16:37:09.508989 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.508969    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"]
May 25 16:37:09.511227 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:37:09.511199    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14a16fa6_ae30_45ac_a835_311d9a99e2c3.slice/crio-edfe2f3f2624f1672f991f1003ca1627c6e7b5ba6b18ff60c866af1ca7bedf01 WatchSource:0}: Error finding container edfe2f3f2624f1672f991f1003ca1627c6e7b5ba6b18ff60c866af1ca7bedf01: Status 404 returned error can't find the container with id edfe2f3f2624f1672f991f1003ca1627c6e7b5ba6b18ff60c866af1ca7bedf01
May 25 16:37:09.512045 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:37:09.512020    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65ea9255_03e8_4801_b096_3e2b106c5bd1.slice/crio-5db5bcd173f2b6f4cb0355ef9f538c61aa1e1dd9fc056934f49981163f2aa3a8 WatchSource:0}: Error finding container 5db5bcd173f2b6f4cb0355ef9f538c61aa1e1dd9fc056934f49981163f2aa3a8: Status 404 returned error can't find the container with id 5db5bcd173f2b6f4cb0355ef9f538c61aa1e1dd9fc056934f49981163f2aa3a8
May 25 16:37:09.580841 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.580790    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b" event={"ID":"14a16fa6-ae30-45ac-a835-311d9a99e2c3","Type":"ContainerStarted","Data":"edfe2f3f2624f1672f991f1003ca1627c6e7b5ba6b18ff60c866af1ca7bedf01"}
May 25 16:37:09.581926 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.581897    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl" event={"ID":"65ea9255-03e8-4801-b096-3e2b106c5bd1","Type":"ContainerStarted","Data":"5db5bcd173f2b6f4cb0355ef9f538c61aa1e1dd9fc056934f49981163f2aa3a8"}
May 25 16:37:09.583049 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.583019    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr" event={"ID":"882706e9-6a8b-4f16-8106-f30d26775096","Type":"ContainerStarted","Data":"a03aa33a23d235c21506f69c92d69719dfef4f484d6c18a2c4c63e3fbfc87559"}
May 25 16:37:09.721131 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.721092    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:09.721131 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.721144    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:09.721377 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.721200    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert\") pod \"ingress-canary-l2p4k\" (UID: \"7a41c3db-9f6d-4e76-9b35-839fa3b35665\") " pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:37:09.721377 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:09.721263    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-jsgsj\" (UID: \"02a5243d-f581-40f8-a48a-b2cfd2296e89\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:37:09.721377 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.721279    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 16:37:09.721377 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.721303    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74bf8cf786-frqqw: secret "image-registry-tls" not found
May 25 16:37:09.721377 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.721327    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 16:37:09.721377 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.721367    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls podName:5ba6020a-e04d-4c62-836f-058b291b1d4b nodeName:}" failed. No retries permitted until 2026-05-25 16:37:10.721345547 +0000 UTC m=+36.069263782 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls") pod "image-registry-74bf8cf786-frqqw" (UID: "5ba6020a-e04d-4c62-836f-058b291b1d4b") : secret "image-registry-tls" not found
May 25 16:37:09.721377 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.721381    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 25 16:37:09.721640 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.721405    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 16:37:09.721640 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.721386    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls podName:dba85d0a-072c-4a30-b169-92843a576b3b nodeName:}" failed. No retries permitted until 2026-05-25 16:37:10.721376755 +0000 UTC m=+36.069294988 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls") pod "dns-default-lndmn" (UID: "dba85d0a-072c-4a30-b169-92843a576b3b") : secret "dns-default-metrics-tls" not found
May 25 16:37:09.721640 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.721440    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert podName:02a5243d-f581-40f8-a48a-b2cfd2296e89 nodeName:}" failed. No retries permitted until 2026-05-25 16:37:10.721424421 +0000 UTC m=+36.069342657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-jsgsj" (UID: "02a5243d-f581-40f8-a48a-b2cfd2296e89") : secret "networking-console-plugin-cert" not found
May 25 16:37:09.721640 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:09.721456    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert podName:7a41c3db-9f6d-4e76-9b35-839fa3b35665 nodeName:}" failed. No retries permitted until 2026-05-25 16:37:10.721446401 +0000 UTC m=+36.069364637 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert") pod "ingress-canary-l2p4k" (UID: "7a41c3db-9f6d-4e76-9b35-839fa3b35665") : secret "canary-serving-cert" not found
May 25 16:37:10.262587 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:10.262551    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:37:10.263839 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:10.262988    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:37:10.263839 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:10.263128    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:37:10.268730 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:10.267530    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 25 16:37:10.268730 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:10.267771    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6d4xp\""
May 25 16:37:10.268730 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:10.267979    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 25 16:37:10.268730 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:10.268171    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 25 16:37:10.268730 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:10.268346    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pmkzv\""
May 25 16:37:10.268730 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:10.268558    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 25 16:37:10.731394 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:10.731306    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert\") pod \"ingress-canary-l2p4k\" (UID: \"7a41c3db-9f6d-4e76-9b35-839fa3b35665\") " pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:37:10.731394 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:10.731386    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-jsgsj\" (UID: \"02a5243d-f581-40f8-a48a-b2cfd2296e89\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:37:10.731624 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:10.731440    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:10.731624 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:10.731475    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:10.731624 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:10.731610    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 16:37:10.732161 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:10.731671    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls podName:dba85d0a-072c-4a30-b169-92843a576b3b nodeName:}" failed. No retries permitted until 2026-05-25 16:37:12.731649336 +0000 UTC m=+38.079567569 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls") pod "dns-default-lndmn" (UID: "dba85d0a-072c-4a30-b169-92843a576b3b") : secret "dns-default-metrics-tls" not found
May 25 16:37:10.732161 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:10.732095    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 16:37:10.732161 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:10.732143    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert podName:7a41c3db-9f6d-4e76-9b35-839fa3b35665 nodeName:}" failed. No retries permitted until 2026-05-25 16:37:12.732128088 +0000 UTC m=+38.080046324 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert") pod "ingress-canary-l2p4k" (UID: "7a41c3db-9f6d-4e76-9b35-839fa3b35665") : secret "canary-serving-cert" not found
May 25 16:37:10.732335 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:10.732200    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 25 16:37:10.732335 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:10.732232    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert podName:02a5243d-f581-40f8-a48a-b2cfd2296e89 nodeName:}" failed. No retries permitted until 2026-05-25 16:37:12.732221746 +0000 UTC m=+38.080139978 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-jsgsj" (UID: "02a5243d-f581-40f8-a48a-b2cfd2296e89") : secret "networking-console-plugin-cert" not found
May 25 16:37:10.732335 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:10.732295    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 16:37:10.732335 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:10.732305    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74bf8cf786-frqqw: secret "image-registry-tls" not found
May 25 16:37:10.732335 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:10.732335    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls podName:5ba6020a-e04d-4c62-836f-058b291b1d4b nodeName:}" failed. No retries permitted until 2026-05-25 16:37:12.73232522 +0000 UTC m=+38.080243451 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls") pod "image-registry-74bf8cf786-frqqw" (UID: "5ba6020a-e04d-4c62-836f-058b291b1d4b") : secret "image-registry-tls" not found
May 25 16:37:12.752077 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:12.751834    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-jsgsj\" (UID: \"02a5243d-f581-40f8-a48a-b2cfd2296e89\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:37:12.752547 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:12.752129    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:12.752547 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:12.751992    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 25 16:37:12.752547 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:12.752167    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:12.752547 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:12.752237    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert podName:02a5243d-f581-40f8-a48a-b2cfd2296e89 nodeName:}" failed. No retries permitted until 2026-05-25 16:37:16.752211652 +0000 UTC m=+42.100129900 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-jsgsj" (UID: "02a5243d-f581-40f8-a48a-b2cfd2296e89") : secret "networking-console-plugin-cert" not found
May 25 16:37:12.752547 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:12.752278    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert\") pod \"ingress-canary-l2p4k\" (UID: \"7a41c3db-9f6d-4e76-9b35-839fa3b35665\") " pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:37:12.752547 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:12.752292    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 16:37:12.752547 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:12.752312    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 16:37:12.752547 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:12.752330    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74bf8cf786-frqqw: secret "image-registry-tls" not found
May 25 16:37:12.752547 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:12.752343    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls podName:dba85d0a-072c-4a30-b169-92843a576b3b nodeName:}" failed. No retries permitted until 2026-05-25 16:37:16.752327624 +0000 UTC m=+42.100245873 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls") pod "dns-default-lndmn" (UID: "dba85d0a-072c-4a30-b169-92843a576b3b") : secret "dns-default-metrics-tls" not found
May 25 16:37:12.752547 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:12.752372    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls podName:5ba6020a-e04d-4c62-836f-058b291b1d4b nodeName:}" failed. No retries permitted until 2026-05-25 16:37:16.752359005 +0000 UTC m=+42.100277250 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls") pod "image-registry-74bf8cf786-frqqw" (UID: "5ba6020a-e04d-4c62-836f-058b291b1d4b") : secret "image-registry-tls" not found
May 25 16:37:12.752547 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:12.752411    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 16:37:12.752547 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:12.752445    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert podName:7a41c3db-9f6d-4e76-9b35-839fa3b35665 nodeName:}" failed. No retries permitted until 2026-05-25 16:37:16.752434129 +0000 UTC m=+42.100352361 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert") pod "ingress-canary-l2p4k" (UID: "7a41c3db-9f6d-4e76-9b35-839fa3b35665") : secret "canary-serving-cert" not found
May 25 16:37:15.598506 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:15.598465    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n7gn7" event={"ID":"6cc91204-a4f1-4108-b4d6-dab40dbda044","Type":"ContainerStarted","Data":"e9504685bf44f59ef251ce9a3628043149d0db819b7d724d17487eddb46a081e"}
May 25 16:37:16.602741 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:16.602703    2554 generic.go:358] "Generic (PLEG): container finished" podID="6cc91204-a4f1-4108-b4d6-dab40dbda044" containerID="e9504685bf44f59ef251ce9a3628043149d0db819b7d724d17487eddb46a081e" exitCode=0
May 25 16:37:16.603141 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:16.602795    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n7gn7" event={"ID":"6cc91204-a4f1-4108-b4d6-dab40dbda044","Type":"ContainerDied","Data":"e9504685bf44f59ef251ce9a3628043149d0db819b7d724d17487eddb46a081e"}
May 25 16:37:16.790216 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:16.790171    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:16.790451 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:16.790231    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:16.790451 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:16.790340    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 16:37:16.790451 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:16.790368    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74bf8cf786-frqqw: secret "image-registry-tls" not found
May 25 16:37:16.790451 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:16.790383    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert\") pod \"ingress-canary-l2p4k\" (UID: \"7a41c3db-9f6d-4e76-9b35-839fa3b35665\") " pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:37:16.790451 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:16.790448    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls podName:5ba6020a-e04d-4c62-836f-058b291b1d4b nodeName:}" failed. No retries permitted until 2026-05-25 16:37:24.79042551 +0000 UTC m=+50.138343762 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls") pod "image-registry-74bf8cf786-frqqw" (UID: "5ba6020a-e04d-4c62-836f-058b291b1d4b") : secret "image-registry-tls" not found
May 25 16:37:16.790711 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:16.790345    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 16:37:16.790711 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:16.790492    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 16:37:16.790711 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:16.790524    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-jsgsj\" (UID: \"02a5243d-f581-40f8-a48a-b2cfd2296e89\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:37:16.790711 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:16.790532    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls podName:dba85d0a-072c-4a30-b169-92843a576b3b nodeName:}" failed. No retries permitted until 2026-05-25 16:37:24.790513715 +0000 UTC m=+50.138431945 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls") pod "dns-default-lndmn" (UID: "dba85d0a-072c-4a30-b169-92843a576b3b") : secret "dns-default-metrics-tls" not found
May 25 16:37:16.790711 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:16.790607    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 25 16:37:16.790711 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:16.790631    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert podName:7a41c3db-9f6d-4e76-9b35-839fa3b35665 nodeName:}" failed. No retries permitted until 2026-05-25 16:37:24.790605662 +0000 UTC m=+50.138523894 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert") pod "ingress-canary-l2p4k" (UID: "7a41c3db-9f6d-4e76-9b35-839fa3b35665") : secret "canary-serving-cert" not found
May 25 16:37:16.790711 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:16.790661    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert podName:02a5243d-f581-40f8-a48a-b2cfd2296e89 nodeName:}" failed. No retries permitted until 2026-05-25 16:37:24.790648179 +0000 UTC m=+50.138566411 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-jsgsj" (UID: "02a5243d-f581-40f8-a48a-b2cfd2296e89") : secret "networking-console-plugin-cert" not found
May 25 16:37:19.610075 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:19.609975    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b" event={"ID":"14a16fa6-ae30-45ac-a835-311d9a99e2c3","Type":"ContainerStarted","Data":"2299f5272efb52b837a00d1e70e13aa515b3a8fc0655f5222bdea143a0b5224a"}
May 25 16:37:19.611164 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:19.611137    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl" event={"ID":"65ea9255-03e8-4801-b096-3e2b106c5bd1","Type":"ContainerStarted","Data":"af0f42b7a6e6455dd9e518a19f0cccd13b9e4d0996a03c70f4e238c9b791b5ad"}
May 25 16:37:19.611352 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:19.611337    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"
May 25 16:37:19.612557 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:19.612530    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr" event={"ID":"882706e9-6a8b-4f16-8106-f30d26775096","Type":"ContainerStarted","Data":"dd4e889b64716c84bf4a95761c7b2a22baaef1133e57f03b51f1a1e059bf1089"}
May 25 16:37:19.613245 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:19.613201    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"
May 25 16:37:19.615327 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:19.615305    2554 generic.go:358] "Generic (PLEG): container finished" podID="6cc91204-a4f1-4108-b4d6-dab40dbda044" containerID="0c4fdb4ba0a0ee416a89cde7f03df122cca236233803f9c88e787e0073ea0326" exitCode=0
May 25 16:37:19.615423 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:19.615344    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n7gn7" event={"ID":"6cc91204-a4f1-4108-b4d6-dab40dbda044","Type":"ContainerDied","Data":"0c4fdb4ba0a0ee416a89cde7f03df122cca236233803f9c88e787e0073ea0326"}
May 25 16:37:19.627645 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:19.627603    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b" podStartSLOduration=3.627295767 podStartE2EDuration="12.627590672s" podCreationTimestamp="2026-05-25 16:37:07 +0000 UTC" firstStartedPulling="2026-05-25 16:37:09.513467932 +0000 UTC m=+34.861386161" lastFinishedPulling="2026-05-25 16:37:18.513762816 +0000 UTC m=+43.861681066" observedRunningTime="2026-05-25 16:37:19.626818113 +0000 UTC m=+44.974736365" watchObservedRunningTime="2026-05-25 16:37:19.627590672 +0000 UTC m=+44.975508923"
May 25 16:37:19.666223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:19.666156    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl" podStartSLOduration=3.06192209 podStartE2EDuration="12.666135122s" podCreationTimestamp="2026-05-25 16:37:07 +0000 UTC" firstStartedPulling="2026-05-25 16:37:09.513715397 +0000 UTC m=+34.861633626" lastFinishedPulling="2026-05-25 16:37:19.117928426 +0000 UTC m=+44.465846658" observedRunningTime="2026-05-25 16:37:19.665356923 +0000 UTC m=+45.013275174" watchObservedRunningTime="2026-05-25 16:37:19.666135122 +0000 UTC m=+45.014053374"
May 25 16:37:20.620290 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:20.620253    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n7gn7" event={"ID":"6cc91204-a4f1-4108-b4d6-dab40dbda044","Type":"ContainerStarted","Data":"001587dfa31102840a34a299b5199de0686530b1e010617988a50eb8a881c336"}
May 25 16:37:20.646885 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:20.646803    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-n7gn7" podStartSLOduration=6.838225164 podStartE2EDuration="45.646784616s" podCreationTimestamp="2026-05-25 16:36:35 +0000 UTC" firstStartedPulling="2026-05-25 16:36:36.597776658 +0000 UTC m=+1.945694886" lastFinishedPulling="2026-05-25 16:37:15.406336094 +0000 UTC m=+40.754254338" observedRunningTime="2026-05-25 16:37:20.645481201 +0000 UTC m=+45.993399452" watchObservedRunningTime="2026-05-25 16:37:20.646784616 +0000 UTC m=+45.994702868"
May 25 16:37:23.627347 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:23.627310    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr" event={"ID":"882706e9-6a8b-4f16-8106-f30d26775096","Type":"ContainerStarted","Data":"7d312bb80deb43d52f1b39af8f3fce5d697971557cc51115e68112629ccd4a81"}
May 25 16:37:23.627347 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:23.627354    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr" event={"ID":"882706e9-6a8b-4f16-8106-f30d26775096","Type":"ContainerStarted","Data":"788b52994c8692c630e0a6ba6d1d522a69fc9934213dd229f70dbc4c7ff98a83"}
May 25 16:37:23.649331 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:23.649286    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr" podStartSLOduration=3.321238806 podStartE2EDuration="16.649272991s" podCreationTimestamp="2026-05-25 16:37:07 +0000 UTC" firstStartedPulling="2026-05-25 16:37:09.508284872 +0000 UTC m=+34.856203113" lastFinishedPulling="2026-05-25 16:37:22.836319067 +0000 UTC m=+48.184237298" observedRunningTime="2026-05-25 16:37:23.648183432 +0000 UTC m=+48.996101683" watchObservedRunningTime="2026-05-25 16:37:23.649272991 +0000 UTC m=+48.997191239"
May 25 16:37:24.860820 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:24.860786    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:24.860820 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:24.860827    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:24.861304 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:24.860883    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert\") pod \"ingress-canary-l2p4k\" (UID: \"7a41c3db-9f6d-4e76-9b35-839fa3b35665\") " pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:37:24.861304 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:24.860934    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-jsgsj\" (UID: \"02a5243d-f581-40f8-a48a-b2cfd2296e89\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:37:24.861304 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:24.860977    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 16:37:24.861304 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:24.860981    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 16:37:24.861304 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:24.861009    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74bf8cf786-frqqw: secret "image-registry-tls" not found
May 25 16:37:24.861304 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:24.861028    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 25 16:37:24.861304 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:24.861051    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls podName:dba85d0a-072c-4a30-b169-92843a576b3b nodeName:}" failed. No retries permitted until 2026-05-25 16:37:40.861027305 +0000 UTC m=+66.208945546 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls") pod "dns-default-lndmn" (UID: "dba85d0a-072c-4a30-b169-92843a576b3b") : secret "dns-default-metrics-tls" not found
May 25 16:37:24.861304 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:24.861070    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls podName:5ba6020a-e04d-4c62-836f-058b291b1d4b nodeName:}" failed. No retries permitted until 2026-05-25 16:37:40.861061086 +0000 UTC m=+66.208979317 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls") pod "image-registry-74bf8cf786-frqqw" (UID: "5ba6020a-e04d-4c62-836f-058b291b1d4b") : secret "image-registry-tls" not found
May 25 16:37:24.861304 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:24.861081    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert podName:02a5243d-f581-40f8-a48a-b2cfd2296e89 nodeName:}" failed. No retries permitted until 2026-05-25 16:37:40.861075198 +0000 UTC m=+66.208993427 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-jsgsj" (UID: "02a5243d-f581-40f8-a48a-b2cfd2296e89") : secret "networking-console-plugin-cert" not found
May 25 16:37:24.861304 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:24.861028    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 16:37:24.861304 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:24.861158    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert podName:7a41c3db-9f6d-4e76-9b35-839fa3b35665 nodeName:}" failed. No retries permitted until 2026-05-25 16:37:40.861139326 +0000 UTC m=+66.209057560 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert") pod "ingress-canary-l2p4k" (UID: "7a41c3db-9f6d-4e76-9b35-839fa3b35665") : secret "canary-serving-cert" not found
May 25 16:37:34.586168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:34.586133    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ggvbg"
May 25 16:37:39.985841 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:39.985780    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret\") pod \"global-pull-secret-syncer-bdqw9\" (UID: \"eb80c465-3f53-4d73-abef-d8465200fa2c\") " pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:37:39.989239 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:39.989219    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 25 16:37:39.999822 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:39.999791    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb80c465-3f53-4d73-abef-d8465200fa2c-original-pull-secret\") pod \"global-pull-secret-syncer-bdqw9\" (UID: \"eb80c465-3f53-4d73-abef-d8465200fa2c\") " pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:37:40.086840 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.086783    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs\") pod \"network-metrics-daemon-l57q2\" (UID: \"6646a861-65d0-4b6f-af0c-415ff46851dd\") " pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:37:40.087059 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.086935    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g7tfc\" (UniqueName: \"kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc\") pod \"network-check-target-fxfkl\" (UID: \"11062d63-8afe-498a-8186-ba3987ab73d4\") " pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:37:40.090775 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.090753    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 25 16:37:40.090909 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.090817    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 25 16:37:40.097469 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:40.097451    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 25 16:37:40.097568 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:40.097522    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs podName:6646a861-65d0-4b6f-af0c-415ff46851dd nodeName:}" failed. No retries permitted until 2026-05-25 16:38:44.097492849 +0000 UTC m=+129.445411082 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs") pod "network-metrics-daemon-l57q2" (UID: "6646a861-65d0-4b6f-af0c-415ff46851dd") : secret "metrics-daemon-secret" not found
May 25 16:37:40.100150 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.100129    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 25 16:37:40.109990 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.109971    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7tfc\" (UniqueName: \"kubernetes.io/projected/11062d63-8afe-498a-8186-ba3987ab73d4-kube-api-access-g7tfc\") pod \"network-check-target-fxfkl\" (UID: \"11062d63-8afe-498a-8186-ba3987ab73d4\") " pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:37:40.282256 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.282210    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdqw9"
May 25 16:37:40.330667 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.330642    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6d4xp\""
May 25 16:37:40.337614 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.337586    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:37:40.413240 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.413204    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bdqw9"]
May 25 16:37:40.417374 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:37:40.417343    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb80c465_3f53_4d73_abef_d8465200fa2c.slice/crio-7a6d735abc6a7aa8fd0f4ef25d1e72a1169668ccb98503502db03c4d4cc3e989 WatchSource:0}: Error finding container 7a6d735abc6a7aa8fd0f4ef25d1e72a1169668ccb98503502db03c4d4cc3e989: Status 404 returned error can't find the container with id 7a6d735abc6a7aa8fd0f4ef25d1e72a1169668ccb98503502db03c4d4cc3e989
May 25 16:37:40.469229 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.469195    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fxfkl"]
May 25 16:37:40.472317 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:37:40.472278    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11062d63_8afe_498a_8186_ba3987ab73d4.slice/crio-f690c6829748946d72e0403c098cd314cbe50fd59790869f5e5daef4056dbbbb WatchSource:0}: Error finding container f690c6829748946d72e0403c098cd314cbe50fd59790869f5e5daef4056dbbbb: Status 404 returned error can't find the container with id f690c6829748946d72e0403c098cd314cbe50fd59790869f5e5daef4056dbbbb
May 25 16:37:40.659358 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.659261    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fxfkl" event={"ID":"11062d63-8afe-498a-8186-ba3987ab73d4","Type":"ContainerStarted","Data":"f690c6829748946d72e0403c098cd314cbe50fd59790869f5e5daef4056dbbbb"}
May 25 16:37:40.660241 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.660211    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bdqw9" event={"ID":"eb80c465-3f53-4d73-abef-d8465200fa2c","Type":"ContainerStarted","Data":"7a6d735abc6a7aa8fd0f4ef25d1e72a1169668ccb98503502db03c4d4cc3e989"}
May 25 16:37:40.894081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.894045    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:37:40.894081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.894085    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:37:40.894282 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.894131    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert\") pod \"ingress-canary-l2p4k\" (UID: \"7a41c3db-9f6d-4e76-9b35-839fa3b35665\") " pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:37:40.894282 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:40.894159    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-jsgsj\" (UID: \"02a5243d-f581-40f8-a48a-b2cfd2296e89\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:37:40.894282 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:40.894199    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 16:37:40.894282 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:40.894223    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74bf8cf786-frqqw: secret "image-registry-tls" not found
May 25 16:37:40.894282 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:40.894266    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 16:37:40.894426 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:40.894295    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls podName:5ba6020a-e04d-4c62-836f-058b291b1d4b nodeName:}" failed. No retries permitted until 2026-05-25 16:38:12.894268485 +0000 UTC m=+98.242186715 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls") pod "image-registry-74bf8cf786-frqqw" (UID: "5ba6020a-e04d-4c62-836f-058b291b1d4b") : secret "image-registry-tls" not found
May 25 16:37:40.894426 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:40.894266    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 16:37:40.894426 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:40.894305    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 25 16:37:40.894426 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:40.894309    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert podName:7a41c3db-9f6d-4e76-9b35-839fa3b35665 nodeName:}" failed. No retries permitted until 2026-05-25 16:38:12.894303284 +0000 UTC m=+98.242221513 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert") pod "ingress-canary-l2p4k" (UID: "7a41c3db-9f6d-4e76-9b35-839fa3b35665") : secret "canary-serving-cert" not found
May 25 16:37:40.894426 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:40.894356    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls podName:dba85d0a-072c-4a30-b169-92843a576b3b nodeName:}" failed. No retries permitted until 2026-05-25 16:38:12.894342472 +0000 UTC m=+98.242260704 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls") pod "dns-default-lndmn" (UID: "dba85d0a-072c-4a30-b169-92843a576b3b") : secret "dns-default-metrics-tls" not found
May 25 16:37:40.894426 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:37:40.894365    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert podName:02a5243d-f581-40f8-a48a-b2cfd2296e89 nodeName:}" failed. No retries permitted until 2026-05-25 16:38:12.894360055 +0000 UTC m=+98.242278283 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-jsgsj" (UID: "02a5243d-f581-40f8-a48a-b2cfd2296e89") : secret "networking-console-plugin-cert" not found
May 25 16:37:45.675320 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:45.675284    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fxfkl" event={"ID":"11062d63-8afe-498a-8186-ba3987ab73d4","Type":"ContainerStarted","Data":"36b4926956514733da28dbb0873408726834dad6a97ded3d28d69cc8952789df"}
May 25 16:37:45.675758 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:45.675452    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:37:45.676615 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:45.676590    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bdqw9" event={"ID":"eb80c465-3f53-4d73-abef-d8465200fa2c","Type":"ContainerStarted","Data":"6bf68a383fa5adea64eb074c46353bd954b2b5bc6bd8833a4f5cab3097ae385b"}
May 25 16:37:45.692135 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:45.692086    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fxfkl" podStartSLOduration=66.368259603 podStartE2EDuration="1m10.69207198s" podCreationTimestamp="2026-05-25 16:36:35 +0000 UTC" firstStartedPulling="2026-05-25 16:37:40.474211547 +0000 UTC m=+65.822129776" lastFinishedPulling="2026-05-25 16:37:44.798023923 +0000 UTC m=+70.145942153" observedRunningTime="2026-05-25 16:37:45.691363515 +0000 UTC m=+71.039281777" watchObservedRunningTime="2026-05-25 16:37:45.69207198 +0000 UTC m=+71.039990232"
May 25 16:37:45.706319 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:37:45.706263    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bdqw9" podStartSLOduration=66.322676204 podStartE2EDuration="1m10.706245047s" podCreationTimestamp="2026-05-25 16:36:35 +0000 UTC" firstStartedPulling="2026-05-25 16:37:40.419610437 +0000 UTC m=+65.767528671" lastFinishedPulling="2026-05-25 16:37:44.80317928 +0000 UTC m=+70.151097514" observedRunningTime="2026-05-25 16:37:45.705917505 +0000 UTC m=+71.053835756" watchObservedRunningTime="2026-05-25 16:37:45.706245047 +0000 UTC m=+71.054163304"
May 25 16:38:12.956385 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:12.956343    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert\") pod \"ingress-canary-l2p4k\" (UID: \"7a41c3db-9f6d-4e76-9b35-839fa3b35665\") " pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:38:12.956385 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:12.956398    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-jsgsj\" (UID: \"02a5243d-f581-40f8-a48a-b2cfd2296e89\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:38:12.957053 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:38:12.956482    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 25 16:38:12.957053 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:38:12.956484    2554 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 16:38:12.957053 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:38:12.956535    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert podName:02a5243d-f581-40f8-a48a-b2cfd2296e89 nodeName:}" failed. No retries permitted until 2026-05-25 16:39:16.956520073 +0000 UTC m=+162.304438301 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-jsgsj" (UID: "02a5243d-f581-40f8-a48a-b2cfd2296e89") : secret "networking-console-plugin-cert" not found
May 25 16:38:12.957053 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:12.956549    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:38:12.957053 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:12.956573    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:38:12.957053 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:38:12.956614    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert podName:7a41c3db-9f6d-4e76-9b35-839fa3b35665 nodeName:}" failed. No retries permitted until 2026-05-25 16:39:16.956596155 +0000 UTC m=+162.304514387 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert") pod "ingress-canary-l2p4k" (UID: "7a41c3db-9f6d-4e76-9b35-839fa3b35665") : secret "canary-serving-cert" not found
May 25 16:38:12.957053 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:38:12.956642    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 16:38:12.957053 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:38:12.956675    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls podName:dba85d0a-072c-4a30-b169-92843a576b3b nodeName:}" failed. No retries permitted until 2026-05-25 16:39:16.956666997 +0000 UTC m=+162.304585226 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls") pod "dns-default-lndmn" (UID: "dba85d0a-072c-4a30-b169-92843a576b3b") : secret "dns-default-metrics-tls" not found
May 25 16:38:12.957053 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:38:12.956692    2554 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 16:38:12.957053 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:38:12.956713    2554 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74bf8cf786-frqqw: secret "image-registry-tls" not found
May 25 16:38:12.957053 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:38:12.956776    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls podName:5ba6020a-e04d-4c62-836f-058b291b1d4b nodeName:}" failed. No retries permitted until 2026-05-25 16:39:16.956761841 +0000 UTC m=+162.304680070 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls") pod "image-registry-74bf8cf786-frqqw" (UID: "5ba6020a-e04d-4c62-836f-058b291b1d4b") : secret "image-registry-tls" not found
May 25 16:38:16.683318 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:16.683283    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fxfkl"
May 25 16:38:44.107067 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:44.107026    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs\") pod \"network-metrics-daemon-l57q2\" (UID: \"6646a861-65d0-4b6f-af0c-415ff46851dd\") " pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:38:44.107561 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:38:44.107138    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 25 16:38:44.107561 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:38:44.107199    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs podName:6646a861-65d0-4b6f-af0c-415ff46851dd nodeName:}" failed. No retries permitted until 2026-05-25 16:40:46.107183718 +0000 UTC m=+251.455101947 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs") pod "network-metrics-daemon-l57q2" (UID: "6646a861-65d0-4b6f-af0c-415ff46851dd") : secret "metrics-daemon-secret" not found
May 25 16:38:56.199933 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:56.199897    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-xx9hh"]
May 25 16:38:56.202741 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:56.202723    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-xx9hh"
May 25 16:38:56.205845 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:56.205819    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-7c29v\""
May 25 16:38:56.219719 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:56.219688    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-xx9hh"]
May 25 16:38:56.304215 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:56.304177    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mt65\" (UniqueName: \"kubernetes.io/projected/d1d39fb0-c3be-49fb-a9e8-78faed5d1df7-kube-api-access-2mt65\") pod \"network-check-source-7f44b6dd87-xx9hh\" (UID: \"d1d39fb0-c3be-49fb-a9e8-78faed5d1df7\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-xx9hh"
May 25 16:38:56.405326 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:56.405274    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2mt65\" (UniqueName: \"kubernetes.io/projected/d1d39fb0-c3be-49fb-a9e8-78faed5d1df7-kube-api-access-2mt65\") pod \"network-check-source-7f44b6dd87-xx9hh\" (UID: \"d1d39fb0-c3be-49fb-a9e8-78faed5d1df7\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-xx9hh"
May 25 16:38:56.414625 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:56.414582    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mt65\" (UniqueName: \"kubernetes.io/projected/d1d39fb0-c3be-49fb-a9e8-78faed5d1df7-kube-api-access-2mt65\") pod \"network-check-source-7f44b6dd87-xx9hh\" (UID: \"d1d39fb0-c3be-49fb-a9e8-78faed5d1df7\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-xx9hh"
May 25 16:38:56.511131 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:56.511090    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-xx9hh"
May 25 16:38:56.631140 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:56.631112    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-xx9hh"]
May 25 16:38:56.634459 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:38:56.634428    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d39fb0_c3be_49fb_a9e8_78faed5d1df7.slice/crio-80416d6dec19abb2b2f770da51e855d0de8cfc08d3d5b244f80ba08010297aa0 WatchSource:0}: Error finding container 80416d6dec19abb2b2f770da51e855d0de8cfc08d3d5b244f80ba08010297aa0: Status 404 returned error can't find the container with id 80416d6dec19abb2b2f770da51e855d0de8cfc08d3d5b244f80ba08010297aa0
May 25 16:38:56.843353 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:56.843270    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-xx9hh" event={"ID":"d1d39fb0-c3be-49fb-a9e8-78faed5d1df7","Type":"ContainerStarted","Data":"8b117e4e076bf011420fe0fc860242b42ef03c94a03f703dd8162e6d3adcd498"}
May 25 16:38:56.843353 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:56.843305    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-xx9hh" event={"ID":"d1d39fb0-c3be-49fb-a9e8-78faed5d1df7","Type":"ContainerStarted","Data":"80416d6dec19abb2b2f770da51e855d0de8cfc08d3d5b244f80ba08010297aa0"}
May 25 16:38:56.859353 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:38:56.859308    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-xx9hh" podStartSLOduration=0.859294024 podStartE2EDuration="859.294024ms" podCreationTimestamp="2026-05-25 16:38:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 16:38:56.858690881 +0000 UTC m=+142.206609132" watchObservedRunningTime="2026-05-25 16:38:56.859294024 +0000 UTC m=+142.207212274"
May 25 16:39:03.896449 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:03.896422    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bx8kp_645b9dd9-5a8a-406b-a2af-5832530e4664/dns-node-resolver/0.log"
May 25 16:39:05.096224 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:05.096198    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wp4m6_d15b78cb-69ab-4017-8bae-92669f025b54/node-ca/0.log"
May 25 16:39:11.981559 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:39:11.981514    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj" podUID="02a5243d-f581-40f8-a48a-b2cfd2296e89"
May 25 16:39:11.992653 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:39:11.992617    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-74bf8cf786-frqqw" podUID="5ba6020a-e04d-4c62-836f-058b291b1d4b"
May 25 16:39:12.040214 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:39:12.040169    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-lndmn" podUID="dba85d0a-072c-4a30-b169-92843a576b3b"
May 25 16:39:12.046382 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:39:12.046350    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-l2p4k" podUID="7a41c3db-9f6d-4e76-9b35-839fa3b35665"
May 25 16:39:12.879497 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:12.879464    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lndmn"
May 25 16:39:12.879687 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:12.879464    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:39:12.879687 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:12.879465    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:39:12.879687 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:12.879464    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:39:13.317606 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:39:13.317565    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-l57q2" podUID="6646a861-65d0-4b6f-af0c-415ff46851dd"
May 25 16:39:16.968611 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:16.968568    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-jsgsj\" (UID: \"02a5243d-f581-40f8-a48a-b2cfd2296e89\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:39:16.969196 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:16.968624    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:39:16.969196 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:16.968653    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:39:16.969196 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:16.968708    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert\") pod \"ingress-canary-l2p4k\" (UID: \"7a41c3db-9f6d-4e76-9b35-839fa3b35665\") " pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:39:16.971148 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:16.971112    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls\") pod \"image-registry-74bf8cf786-frqqw\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") " pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:39:16.971253 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:16.971110    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02a5243d-f581-40f8-a48a-b2cfd2296e89-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-jsgsj\" (UID: \"02a5243d-f581-40f8-a48a-b2cfd2296e89\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:39:16.971390 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:16.971375    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a41c3db-9f6d-4e76-9b35-839fa3b35665-cert\") pod \"ingress-canary-l2p4k\" (UID: \"7a41c3db-9f6d-4e76-9b35-839fa3b35665\") " pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:39:16.971431 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:16.971377    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dba85d0a-072c-4a30-b169-92843a576b3b-metrics-tls\") pod \"dns-default-lndmn\" (UID: \"dba85d0a-072c-4a30-b169-92843a576b3b\") " pod="openshift-dns/dns-default-lndmn"
May 25 16:39:17.085083 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.085047    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-f46lz\""
May 25 16:39:17.085260 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.085102    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rqkd2\""
May 25 16:39:17.085260 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.085108    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mbvc2\""
May 25 16:39:17.085260 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.085108    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6lmcz\""
May 25 16:39:17.091405 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.091376    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:39:17.091483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.091419    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lndmn"
May 25 16:39:17.091483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.091394    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l2p4k"
May 25 16:39:17.091556 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.091376    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"
May 25 16:39:17.251683 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.251652    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj"]
May 25 16:39:17.254762 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:39:17.254733    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02a5243d_f581_40f8_a48a_b2cfd2296e89.slice/crio-49d5dbc600e8ae9d0d0cf14bf65621120471c5be4ece3fb24d25b7130c15a447 WatchSource:0}: Error finding container 49d5dbc600e8ae9d0d0cf14bf65621120471c5be4ece3fb24d25b7130c15a447: Status 404 returned error can't find the container with id 49d5dbc600e8ae9d0d0cf14bf65621120471c5be4ece3fb24d25b7130c15a447
May 25 16:39:17.477434 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.477389    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l2p4k"]
May 25 16:39:17.477947 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.477924    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lndmn"]
May 25 16:39:17.480647 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.480626    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74bf8cf786-frqqw"]
May 25 16:39:17.482211 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:39:17.482179    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a41c3db_9f6d_4e76_9b35_839fa3b35665.slice/crio-b95e8584ea2f69819bed25de52979da24005b38773437f22f10c64be561c5738 WatchSource:0}: Error finding container b95e8584ea2f69819bed25de52979da24005b38773437f22f10c64be561c5738: Status 404 returned error can't find the container with id b95e8584ea2f69819bed25de52979da24005b38773437f22f10c64be561c5738
May 25 16:39:17.482996 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:39:17.482973    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba85d0a_072c_4a30_b169_92843a576b3b.slice/crio-b3d1ad93f8ee188b2157116af0b8a14d9424dfa0087100bfe04b4225a7546f4e WatchSource:0}: Error finding container b3d1ad93f8ee188b2157116af0b8a14d9424dfa0087100bfe04b4225a7546f4e: Status 404 returned error can't find the container with id b3d1ad93f8ee188b2157116af0b8a14d9424dfa0087100bfe04b4225a7546f4e
May 25 16:39:17.484105 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:39:17.484082    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba6020a_e04d_4c62_836f_058b291b1d4b.slice/crio-0161b72016605432e58cf7e11c60562e9b5ebaf002ca417b90df609be3bae658 WatchSource:0}: Error finding container 0161b72016605432e58cf7e11c60562e9b5ebaf002ca417b90df609be3bae658: Status 404 returned error can't find the container with id 0161b72016605432e58cf7e11c60562e9b5ebaf002ca417b90df609be3bae658
May 25 16:39:17.892552 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.892464    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lndmn" event={"ID":"dba85d0a-072c-4a30-b169-92843a576b3b","Type":"ContainerStarted","Data":"b3d1ad93f8ee188b2157116af0b8a14d9424dfa0087100bfe04b4225a7546f4e"}
May 25 16:39:17.893964 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.893931    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj" event={"ID":"02a5243d-f581-40f8-a48a-b2cfd2296e89","Type":"ContainerStarted","Data":"49d5dbc600e8ae9d0d0cf14bf65621120471c5be4ece3fb24d25b7130c15a447"}
May 25 16:39:17.895669 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.895635    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74bf8cf786-frqqw" event={"ID":"5ba6020a-e04d-4c62-836f-058b291b1d4b","Type":"ContainerStarted","Data":"ecf9c3f324c8e430854457e4cc21036ab3143436dcc0a0170281e56fd6083cb4"}
May 25 16:39:17.895831 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.895671    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74bf8cf786-frqqw" event={"ID":"5ba6020a-e04d-4c62-836f-058b291b1d4b","Type":"ContainerStarted","Data":"0161b72016605432e58cf7e11c60562e9b5ebaf002ca417b90df609be3bae658"}
May 25 16:39:17.899906 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.896315    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:39:17.901667 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.901639    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l2p4k" event={"ID":"7a41c3db-9f6d-4e76-9b35-839fa3b35665","Type":"ContainerStarted","Data":"b95e8584ea2f69819bed25de52979da24005b38773437f22f10c64be561c5738"}
May 25 16:39:17.917655 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:17.917586    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74bf8cf786-frqqw" podStartSLOduration=162.917565162 podStartE2EDuration="2m42.917565162s" podCreationTimestamp="2026-05-25 16:36:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 16:39:17.916672309 +0000 UTC m=+163.264590558" watchObservedRunningTime="2026-05-25 16:39:17.917565162 +0000 UTC m=+163.265483413"
May 25 16:39:18.906528 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:18.906425    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj" event={"ID":"02a5243d-f581-40f8-a48a-b2cfd2296e89","Type":"ContainerStarted","Data":"b38ea2d8b469da4175274a9873d8331ba2d10615d1b4d247c714a553d8293af9"}
May 25 16:39:18.928942 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:18.928861    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-jsgsj" podStartSLOduration=159.637854246 podStartE2EDuration="2m40.928843587s" podCreationTimestamp="2026-05-25 16:36:38 +0000 UTC" firstStartedPulling="2026-05-25 16:39:17.256640198 +0000 UTC m=+162.604558432" lastFinishedPulling="2026-05-25 16:39:18.547629529 +0000 UTC m=+163.895547773" observedRunningTime="2026-05-25 16:39:18.928753079 +0000 UTC m=+164.276671329" watchObservedRunningTime="2026-05-25 16:39:18.928843587 +0000 UTC m=+164.276761879"
May 25 16:39:19.328649 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:19.328596    2554 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b" podUID="14a16fa6-ae30-45ac-a835-311d9a99e2c3" containerName="addon-agent" probeResult="failure" output="Get \"http://10.132.0.10:8000/healthz\": dial tcp 10.132.0.10:8000: connect: connection refused"
May 25 16:39:19.611984 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:19.611942    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl" podUID="65ea9255-03e8-4801-b096-3e2b106c5bd1" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.9:8000/readyz\": dial tcp 10.132.0.9:8000: connect: connection refused"
May 25 16:39:19.910466 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:19.910377    2554 generic.go:358] "Generic (PLEG): container finished" podID="65ea9255-03e8-4801-b096-3e2b106c5bd1" containerID="af0f42b7a6e6455dd9e518a19f0cccd13b9e4d0996a03c70f4e238c9b791b5ad" exitCode=1
May 25 16:39:19.910466 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:19.910439    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl" event={"ID":"65ea9255-03e8-4801-b096-3e2b106c5bd1","Type":"ContainerDied","Data":"af0f42b7a6e6455dd9e518a19f0cccd13b9e4d0996a03c70f4e238c9b791b5ad"}
May 25 16:39:19.910958 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:19.910810    2554 scope.go:117] "RemoveContainer" containerID="af0f42b7a6e6455dd9e518a19f0cccd13b9e4d0996a03c70f4e238c9b791b5ad"
May 25 16:39:19.911803 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:19.911773    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l2p4k" event={"ID":"7a41c3db-9f6d-4e76-9b35-839fa3b35665","Type":"ContainerStarted","Data":"4fc765f50625535f8194963bc8830d995599cf2f562ee816ded733058842f887"}
May 25 16:39:19.913045 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:19.912980    2554 generic.go:358] "Generic (PLEG): container finished" podID="14a16fa6-ae30-45ac-a835-311d9a99e2c3" containerID="2299f5272efb52b837a00d1e70e13aa515b3a8fc0655f5222bdea143a0b5224a" exitCode=255
May 25 16:39:19.913105 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:19.913048    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b" event={"ID":"14a16fa6-ae30-45ac-a835-311d9a99e2c3","Type":"ContainerDied","Data":"2299f5272efb52b837a00d1e70e13aa515b3a8fc0655f5222bdea143a0b5224a"}
May 25 16:39:19.913329 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:19.913303    2554 scope.go:117] "RemoveContainer" containerID="2299f5272efb52b837a00d1e70e13aa515b3a8fc0655f5222bdea143a0b5224a"
May 25 16:39:19.914656 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:19.914616    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lndmn" event={"ID":"dba85d0a-072c-4a30-b169-92843a576b3b","Type":"ContainerStarted","Data":"2e52db255b957e4715e05de4fbd640abb1d0842a25073acbc587b358fee642a8"}
May 25 16:39:19.914656 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:19.914645    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lndmn" event={"ID":"dba85d0a-072c-4a30-b169-92843a576b3b","Type":"ContainerStarted","Data":"87a7a68fd5d5836406eeca341df120eb9765a88c7b664324990f151043012927"}
May 25 16:39:19.914759 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:19.914714    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lndmn"
May 25 16:39:19.948153 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:19.948074    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-lndmn" podStartSLOduration=129.847261034 podStartE2EDuration="2m11.948049108s" podCreationTimestamp="2026-05-25 16:37:08 +0000 UTC" firstStartedPulling="2026-05-25 16:39:17.484821491 +0000 UTC m=+162.832739720" lastFinishedPulling="2026-05-25 16:39:19.585609562 +0000 UTC m=+164.933527794" observedRunningTime="2026-05-25 16:39:19.947163316 +0000 UTC m=+165.295081569" watchObservedRunningTime="2026-05-25 16:39:19.948049108 +0000 UTC m=+165.295967359"
May 25 16:39:19.981388 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:19.981317    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-l2p4k" podStartSLOduration=129.876542872 podStartE2EDuration="2m11.98129012s" podCreationTimestamp="2026-05-25 16:37:08 +0000 UTC" firstStartedPulling="2026-05-25 16:39:17.484130645 +0000 UTC m=+162.832048875" lastFinishedPulling="2026-05-25 16:39:19.588877875 +0000 UTC m=+164.936796123" observedRunningTime="2026-05-25 16:39:19.980464544 +0000 UTC m=+165.328382794" watchObservedRunningTime="2026-05-25 16:39:19.98129012 +0000 UTC m=+165.329208369"
May 25 16:39:20.918825 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:20.918785    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl" event={"ID":"65ea9255-03e8-4801-b096-3e2b106c5bd1","Type":"ContainerStarted","Data":"28f09f36e594f8712eec98c95ecb6bc6b77fc69bf0a968ec5c924e7eb53820ae"}
May 25 16:39:20.919323 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:20.919157    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"
May 25 16:39:20.919817 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:20.919794    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d678c9b9b-dbqhl"
May 25 16:39:20.920402 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:20.920375    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d54f7fc8-dns4b" event={"ID":"14a16fa6-ae30-45ac-a835-311d9a99e2c3","Type":"ContainerStarted","Data":"5447b95919d31696250637d527cb68834a2b23a6910624c4230ea86701588b75"}
May 25 16:39:23.592079 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.592043    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vmnrg"]
May 25 16:39:23.595058 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.595037    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:23.615009 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.614981    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 25 16:39:23.616011 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.615987    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 25 16:39:23.616152 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.616039    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9zxrl\""
May 25 16:39:23.616617 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.616602    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 25 16:39:23.624133 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.624111    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 25 16:39:23.624908 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.624887    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f99e26e0-ccfa-44a2-87ca-f50d6d75b059-crio-socket\") pod \"insights-runtime-extractor-vmnrg\" (UID: \"f99e26e0-ccfa-44a2-87ca-f50d6d75b059\") " pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:23.625012 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.624930    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f99e26e0-ccfa-44a2-87ca-f50d6d75b059-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmnrg\" (UID: \"f99e26e0-ccfa-44a2-87ca-f50d6d75b059\") " pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:23.625072 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.625002    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f99e26e0-ccfa-44a2-87ca-f50d6d75b059-data-volume\") pod \"insights-runtime-extractor-vmnrg\" (UID: \"f99e26e0-ccfa-44a2-87ca-f50d6d75b059\") " pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:23.625072 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.625049    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvb8l\" (UniqueName: \"kubernetes.io/projected/f99e26e0-ccfa-44a2-87ca-f50d6d75b059-kube-api-access-jvb8l\") pod \"insights-runtime-extractor-vmnrg\" (UID: \"f99e26e0-ccfa-44a2-87ca-f50d6d75b059\") " pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:23.625168 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.625104    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f99e26e0-ccfa-44a2-87ca-f50d6d75b059-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmnrg\" (UID: \"f99e26e0-ccfa-44a2-87ca-f50d6d75b059\") " pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:23.626409 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.626389    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vmnrg"]
May 25 16:39:23.725527 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.725496    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jvb8l\" (UniqueName: \"kubernetes.io/projected/f99e26e0-ccfa-44a2-87ca-f50d6d75b059-kube-api-access-jvb8l\") pod \"insights-runtime-extractor-vmnrg\" (UID: \"f99e26e0-ccfa-44a2-87ca-f50d6d75b059\") " pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:23.725527 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.725539    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f99e26e0-ccfa-44a2-87ca-f50d6d75b059-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmnrg\" (UID: \"f99e26e0-ccfa-44a2-87ca-f50d6d75b059\") " pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:23.725737 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.725601    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f99e26e0-ccfa-44a2-87ca-f50d6d75b059-crio-socket\") pod \"insights-runtime-extractor-vmnrg\" (UID: \"f99e26e0-ccfa-44a2-87ca-f50d6d75b059\") " pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:23.725737 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.725655    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f99e26e0-ccfa-44a2-87ca-f50d6d75b059-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmnrg\" (UID: \"f99e26e0-ccfa-44a2-87ca-f50d6d75b059\") " pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:23.725737 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.725696    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f99e26e0-ccfa-44a2-87ca-f50d6d75b059-data-volume\") pod \"insights-runtime-extractor-vmnrg\" (UID: \"f99e26e0-ccfa-44a2-87ca-f50d6d75b059\") " pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:23.725901 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.725753    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f99e26e0-ccfa-44a2-87ca-f50d6d75b059-crio-socket\") pod \"insights-runtime-extractor-vmnrg\" (UID: \"f99e26e0-ccfa-44a2-87ca-f50d6d75b059\") " pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:23.726022 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.726003    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f99e26e0-ccfa-44a2-87ca-f50d6d75b059-data-volume\") pod \"insights-runtime-extractor-vmnrg\" (UID: \"f99e26e0-ccfa-44a2-87ca-f50d6d75b059\") " pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:23.726263 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.726245    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f99e26e0-ccfa-44a2-87ca-f50d6d75b059-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmnrg\" (UID: \"f99e26e0-ccfa-44a2-87ca-f50d6d75b059\") " pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:23.727785 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.727770    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f99e26e0-ccfa-44a2-87ca-f50d6d75b059-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmnrg\" (UID: \"f99e26e0-ccfa-44a2-87ca-f50d6d75b059\") " pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:23.744984 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.744949    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvb8l\" (UniqueName: \"kubernetes.io/projected/f99e26e0-ccfa-44a2-87ca-f50d6d75b059-kube-api-access-jvb8l\") pod \"insights-runtime-extractor-vmnrg\" (UID: \"f99e26e0-ccfa-44a2-87ca-f50d6d75b059\") " pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:23.904326 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:23.904234    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vmnrg"
May 25 16:39:24.033826 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:24.033790    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vmnrg"]
May 25 16:39:24.038383 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:39:24.038351    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99e26e0_ccfa_44a2_87ca_f50d6d75b059.slice/crio-e572289b5b96e6242fc3e5bb4a350de59bce69ac9e6aecf0635e659e08f0d6ce WatchSource:0}: Error finding container e572289b5b96e6242fc3e5bb4a350de59bce69ac9e6aecf0635e659e08f0d6ce: Status 404 returned error can't find the container with id e572289b5b96e6242fc3e5bb4a350de59bce69ac9e6aecf0635e659e08f0d6ce
May 25 16:39:24.935032 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:24.935001    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vmnrg" event={"ID":"f99e26e0-ccfa-44a2-87ca-f50d6d75b059","Type":"ContainerStarted","Data":"62fb19f07870bc2eef12d8c8e6b62d637910c1a7ef0b351695718826387f7352"}
May 25 16:39:24.935300 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:24.935040    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vmnrg" event={"ID":"f99e26e0-ccfa-44a2-87ca-f50d6d75b059","Type":"ContainerStarted","Data":"e572289b5b96e6242fc3e5bb4a350de59bce69ac9e6aecf0635e659e08f0d6ce"}
May 25 16:39:25.939545 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:25.939508    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vmnrg" event={"ID":"f99e26e0-ccfa-44a2-87ca-f50d6d75b059","Type":"ContainerStarted","Data":"c5a1eecd3cfbda4241da3f4bab5fb9288591c2056521f5993e34cea73f8a17ad"}
May 25 16:39:26.262844 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:26.262800    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:39:26.944027 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:26.943991    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vmnrg" event={"ID":"f99e26e0-ccfa-44a2-87ca-f50d6d75b059","Type":"ContainerStarted","Data":"e64ba864e64ce1db35578812dbe0df8b3657ffebd008016d970f52b4957569d9"}
May 25 16:39:26.965641 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:26.965584    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vmnrg" podStartSLOduration=1.659732482 podStartE2EDuration="3.965568237s" podCreationTimestamp="2026-05-25 16:39:23 +0000 UTC" firstStartedPulling="2026-05-25 16:39:24.090994916 +0000 UTC m=+169.438913144" lastFinishedPulling="2026-05-25 16:39:26.396830671 +0000 UTC m=+171.744748899" observedRunningTime="2026-05-25 16:39:26.964579907 +0000 UTC m=+172.312498157" watchObservedRunningTime="2026-05-25 16:39:26.965568237 +0000 UTC m=+172.313486487"
May 25 16:39:29.922984 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:29.922956    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lndmn"
May 25 16:39:37.095506 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:37.095471    2554 patch_prober.go:28] interesting pod/image-registry-74bf8cf786-frqqw container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 25 16:39:37.095902 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:37.095526    2554 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-74bf8cf786-frqqw" podUID="5ba6020a-e04d-4c62-836f-058b291b1d4b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 25 16:39:37.996701 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:37.996666    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mwjld"]
May 25 16:39:38.000053 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.000033    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.002988 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.002967    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 25 16:39:38.003118 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.002983    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 25 16:39:38.004042 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.004024    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 25 16:39:38.004158 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.004142    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 25 16:39:38.004224 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.004155    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p5vzn\""
May 25 16:39:38.004224 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.004176    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 25 16:39:38.004326 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.004253    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 25 16:39:38.129568 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.129526    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-metrics-client-ca\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.129568 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.129573    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-node-exporter-wtmp\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.130011 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.129607    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.130011 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.129633    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-root\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.130011 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.129671    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-sys\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.130011 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.129719    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-node-exporter-tls\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.130011 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.129751    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-node-exporter-accelerators-collector-config\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.130011 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.129774    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9ztd\" (UniqueName: \"kubernetes.io/projected/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-kube-api-access-m9ztd\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.130011 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.129808    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-node-exporter-textfile\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.230546 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.230504    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-metrics-client-ca\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.230546 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.230549    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-node-exporter-wtmp\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.230811 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.230685    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-node-exporter-wtmp\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.230811 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.230682    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.230811 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.230712    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-root\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.230811 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.230743    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-sys\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.230811 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.230759    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-node-exporter-tls\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.230811 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.230777    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-node-exporter-accelerators-collector-config\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.231131 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.230829    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m9ztd\" (UniqueName: \"kubernetes.io/projected/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-kube-api-access-m9ztd\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.231131 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.230845    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-sys\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.231131 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.230916    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-node-exporter-textfile\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.231131 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.230947    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-root\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.231284 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.231185    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-metrics-client-ca\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.231332 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.231311    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-node-exporter-textfile\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.231374 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.231326    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-node-exporter-accelerators-collector-config\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.233035 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.233017    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.233160 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.233145    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-node-exporter-tls\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.247952 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.247885    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9ztd\" (UniqueName: \"kubernetes.io/projected/51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a-kube-api-access-m9ztd\") pod \"node-exporter-mwjld\" (UID: \"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a\") " pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.308772 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.308737    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mwjld"
May 25 16:39:38.316489 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:39:38.316457    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e96ef1_e2ed_49dc_ab7f_d6bdec2ef69a.slice/crio-9e23f5fb1e0fa316b899a522c593fca25e71bf99f0ddd3e08e11a4c411b88cb4 WatchSource:0}: Error finding container 9e23f5fb1e0fa316b899a522c593fca25e71bf99f0ddd3e08e11a4c411b88cb4: Status 404 returned error can't find the container with id 9e23f5fb1e0fa316b899a522c593fca25e71bf99f0ddd3e08e11a4c411b88cb4
May 25 16:39:38.911506 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.911475    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:39:38.975389 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:38.975342    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mwjld" event={"ID":"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a","Type":"ContainerStarted","Data":"9e23f5fb1e0fa316b899a522c593fca25e71bf99f0ddd3e08e11a4c411b88cb4"}
May 25 16:39:39.979011 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:39.978979    2554 generic.go:358] "Generic (PLEG): container finished" podID="51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a" containerID="0e09d136c24a3022e7f145e7662ef715d16cfada9461c64a1e6ef1afa5dec617" exitCode=0
May 25 16:39:39.979378 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:39.979031    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mwjld" event={"ID":"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a","Type":"ContainerDied","Data":"0e09d136c24a3022e7f145e7662ef715d16cfada9461c64a1e6ef1afa5dec617"}
May 25 16:39:40.984070 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:40.984038    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mwjld" event={"ID":"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a","Type":"ContainerStarted","Data":"9f5fee88ead6c98b102a6ea74a9468621abda9b70c091204ee50627b3f72949e"}
May 25 16:39:40.984070 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:40.984076    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mwjld" event={"ID":"51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a","Type":"ContainerStarted","Data":"58c4c7fcdee942a70aff416d48b5130a5c253d7210c83da80f73383eaef406c6"}
May 25 16:39:41.021473 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:41.021418    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mwjld" podStartSLOduration=2.957465997 podStartE2EDuration="4.021403265s" podCreationTimestamp="2026-05-25 16:39:37 +0000 UTC" firstStartedPulling="2026-05-25 16:39:38.318213733 +0000 UTC m=+183.666131961" lastFinishedPulling="2026-05-25 16:39:39.382150983 +0000 UTC m=+184.730069229" observedRunningTime="2026-05-25 16:39:41.019647698 +0000 UTC m=+186.367565947" watchObservedRunningTime="2026-05-25 16:39:41.021403265 +0000 UTC m=+186.369321516"
May 25 16:39:45.942470 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:39:45.942431    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74bf8cf786-frqqw"]
May 25 16:40:09.311382 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:09.311341    2554 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr" podUID="882706e9-6a8b-4f16-8106-f30d26775096" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 25 16:40:10.960947 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:10.960879    2554 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-74bf8cf786-frqqw" podUID="5ba6020a-e04d-4c62-836f-058b291b1d4b" containerName="registry" containerID="cri-o://ecf9c3f324c8e430854457e4cc21036ab3143436dcc0a0170281e56fd6083cb4" gracePeriod=30
May 25 16:40:11.223366 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.223337    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:40:11.406956 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.406908    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ba6020a-e04d-4c62-836f-058b291b1d4b-image-registry-private-configuration\") pod \"5ba6020a-e04d-4c62-836f-058b291b1d4b\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") "
May 25 16:40:11.406956 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.406962    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ba6020a-e04d-4c62-836f-058b291b1d4b-ca-trust-extracted\") pod \"5ba6020a-e04d-4c62-836f-058b291b1d4b\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") "
May 25 16:40:11.407203 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.406985    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ba6020a-e04d-4c62-836f-058b291b1d4b-trusted-ca\") pod \"5ba6020a-e04d-4c62-836f-058b291b1d4b\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") "
May 25 16:40:11.407203 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.407019    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-bound-sa-token\") pod \"5ba6020a-e04d-4c62-836f-058b291b1d4b\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") "
May 25 16:40:11.407203 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.407050    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-certificates\") pod \"5ba6020a-e04d-4c62-836f-058b291b1d4b\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") "
May 25 16:40:11.407203 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.407074    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls\") pod \"5ba6020a-e04d-4c62-836f-058b291b1d4b\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") "
May 25 16:40:11.407203 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.407098    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzzzm\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-kube-api-access-hzzzm\") pod \"5ba6020a-e04d-4c62-836f-058b291b1d4b\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") "
May 25 16:40:11.407203 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.407125    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ba6020a-e04d-4c62-836f-058b291b1d4b-installation-pull-secrets\") pod \"5ba6020a-e04d-4c62-836f-058b291b1d4b\" (UID: \"5ba6020a-e04d-4c62-836f-058b291b1d4b\") "
May 25 16:40:11.407627 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.407566    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ba6020a-e04d-4c62-836f-058b291b1d4b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5ba6020a-e04d-4c62-836f-058b291b1d4b" (UID: "5ba6020a-e04d-4c62-836f-058b291b1d4b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 16:40:11.407627 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.407583    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5ba6020a-e04d-4c62-836f-058b291b1d4b" (UID: "5ba6020a-e04d-4c62-836f-058b291b1d4b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 16:40:11.409640 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.409597    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ba6020a-e04d-4c62-836f-058b291b1d4b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5ba6020a-e04d-4c62-836f-058b291b1d4b" (UID: "5ba6020a-e04d-4c62-836f-058b291b1d4b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 16:40:11.409771 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.409724    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ba6020a-e04d-4c62-836f-058b291b1d4b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5ba6020a-e04d-4c62-836f-058b291b1d4b" (UID: "5ba6020a-e04d-4c62-836f-058b291b1d4b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 16:40:11.409835 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.409776    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-kube-api-access-hzzzm" (OuterVolumeSpecName: "kube-api-access-hzzzm") pod "5ba6020a-e04d-4c62-836f-058b291b1d4b" (UID: "5ba6020a-e04d-4c62-836f-058b291b1d4b"). InnerVolumeSpecName "kube-api-access-hzzzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 16:40:11.409950 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.409899    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5ba6020a-e04d-4c62-836f-058b291b1d4b" (UID: "5ba6020a-e04d-4c62-836f-058b291b1d4b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 16:40:11.410031 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.410006    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5ba6020a-e04d-4c62-836f-058b291b1d4b" (UID: "5ba6020a-e04d-4c62-836f-058b291b1d4b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 16:40:11.416139 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.416109    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba6020a-e04d-4c62-836f-058b291b1d4b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5ba6020a-e04d-4c62-836f-058b291b1d4b" (UID: "5ba6020a-e04d-4c62-836f-058b291b1d4b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 16:40:11.508296 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.508254    2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ba6020a-e04d-4c62-836f-058b291b1d4b-trusted-ca\") on node \"ip-10-0-135-137.ec2.internal\" DevicePath \"\""
May 25 16:40:11.508296 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.508291    2554 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-bound-sa-token\") on node \"ip-10-0-135-137.ec2.internal\" DevicePath \"\""
May 25 16:40:11.508508 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.508303    2554 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-certificates\") on node \"ip-10-0-135-137.ec2.internal\" DevicePath \"\""
May 25 16:40:11.508508 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.508317    2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-registry-tls\") on node \"ip-10-0-135-137.ec2.internal\" DevicePath \"\""
May 25 16:40:11.508508 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.508328    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzzzm\" (UniqueName: \"kubernetes.io/projected/5ba6020a-e04d-4c62-836f-058b291b1d4b-kube-api-access-hzzzm\") on node \"ip-10-0-135-137.ec2.internal\" DevicePath \"\""
May 25 16:40:11.508508 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.508339    2554 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ba6020a-e04d-4c62-836f-058b291b1d4b-installation-pull-secrets\") on node \"ip-10-0-135-137.ec2.internal\" DevicePath \"\""
May 25 16:40:11.508508 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.508351    2554 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ba6020a-e04d-4c62-836f-058b291b1d4b-image-registry-private-configuration\") on node \"ip-10-0-135-137.ec2.internal\" DevicePath \"\""
May 25 16:40:11.508508 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:11.508364    2554 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ba6020a-e04d-4c62-836f-058b291b1d4b-ca-trust-extracted\") on node \"ip-10-0-135-137.ec2.internal\" DevicePath \"\""
May 25 16:40:12.065755 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:12.065722    2554 generic.go:358] "Generic (PLEG): container finished" podID="5ba6020a-e04d-4c62-836f-058b291b1d4b" containerID="ecf9c3f324c8e430854457e4cc21036ab3143436dcc0a0170281e56fd6083cb4" exitCode=0
May 25 16:40:12.066198 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:12.065780    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74bf8cf786-frqqw"
May 25 16:40:12.066198 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:12.065796    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74bf8cf786-frqqw" event={"ID":"5ba6020a-e04d-4c62-836f-058b291b1d4b","Type":"ContainerDied","Data":"ecf9c3f324c8e430854457e4cc21036ab3143436dcc0a0170281e56fd6083cb4"}
May 25 16:40:12.066198 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:12.065835    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74bf8cf786-frqqw" event={"ID":"5ba6020a-e04d-4c62-836f-058b291b1d4b","Type":"ContainerDied","Data":"0161b72016605432e58cf7e11c60562e9b5ebaf002ca417b90df609be3bae658"}
May 25 16:40:12.066198 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:12.065850    2554 scope.go:117] "RemoveContainer" containerID="ecf9c3f324c8e430854457e4cc21036ab3143436dcc0a0170281e56fd6083cb4"
May 25 16:40:12.074488 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:12.074468    2554 scope.go:117] "RemoveContainer" containerID="ecf9c3f324c8e430854457e4cc21036ab3143436dcc0a0170281e56fd6083cb4"
May 25 16:40:12.074734 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:40:12.074718    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecf9c3f324c8e430854457e4cc21036ab3143436dcc0a0170281e56fd6083cb4\": container with ID starting with ecf9c3f324c8e430854457e4cc21036ab3143436dcc0a0170281e56fd6083cb4 not found: ID does not exist" containerID="ecf9c3f324c8e430854457e4cc21036ab3143436dcc0a0170281e56fd6083cb4"
May 25 16:40:12.074777 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:12.074743    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecf9c3f324c8e430854457e4cc21036ab3143436dcc0a0170281e56fd6083cb4"} err="failed to get container status \"ecf9c3f324c8e430854457e4cc21036ab3143436dcc0a0170281e56fd6083cb4\": rpc error: code = NotFound desc = could not find container \"ecf9c3f324c8e430854457e4cc21036ab3143436dcc0a0170281e56fd6083cb4\": container with ID starting with ecf9c3f324c8e430854457e4cc21036ab3143436dcc0a0170281e56fd6083cb4 not found: ID does not exist"
May 25 16:40:12.086577 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:12.086535    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74bf8cf786-frqqw"]
May 25 16:40:12.092411 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:12.092388    2554 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-74bf8cf786-frqqw"]
May 25 16:40:13.266100 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:13.266067    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ba6020a-e04d-4c62-836f-058b291b1d4b" path="/var/lib/kubelet/pods/5ba6020a-e04d-4c62-836f-058b291b1d4b/volumes"
May 25 16:40:19.311395 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:19.311357    2554 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr" podUID="882706e9-6a8b-4f16-8106-f30d26775096" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 25 16:40:29.311690 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:29.311650    2554 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr" podUID="882706e9-6a8b-4f16-8106-f30d26775096" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 25 16:40:29.312071 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:29.311719    2554 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr"
May 25 16:40:29.312213 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:29.312196    2554 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"7d312bb80deb43d52f1b39af8f3fce5d697971557cc51115e68112629ccd4a81"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr" containerMessage="Container service-proxy failed liveness probe, will be restarted"
May 25 16:40:29.312244 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:29.312232    2554 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr" podUID="882706e9-6a8b-4f16-8106-f30d26775096" containerName="service-proxy" containerID="cri-o://7d312bb80deb43d52f1b39af8f3fce5d697971557cc51115e68112629ccd4a81" gracePeriod=30
May 25 16:40:30.115512 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:30.115479    2554 generic.go:358] "Generic (PLEG): container finished" podID="882706e9-6a8b-4f16-8106-f30d26775096" containerID="7d312bb80deb43d52f1b39af8f3fce5d697971557cc51115e68112629ccd4a81" exitCode=2
May 25 16:40:30.115682 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:30.115550    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr" event={"ID":"882706e9-6a8b-4f16-8106-f30d26775096","Type":"ContainerDied","Data":"7d312bb80deb43d52f1b39af8f3fce5d697971557cc51115e68112629ccd4a81"}
May 25 16:40:30.115682 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:30.115583    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67b7fccbcc-f5czr" event={"ID":"882706e9-6a8b-4f16-8106-f30d26775096","Type":"ContainerStarted","Data":"3d0f487c5d558fdfbfedf85c84435d4c2f4390222090d84fc5ab3aae3575629d"}
May 25 16:40:46.157100 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:46.157062    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs\") pod \"network-metrics-daemon-l57q2\" (UID: \"6646a861-65d0-4b6f-af0c-415ff46851dd\") " pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:40:46.159406 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:46.159380    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6646a861-65d0-4b6f-af0c-415ff46851dd-metrics-certs\") pod \"network-metrics-daemon-l57q2\" (UID: \"6646a861-65d0-4b6f-af0c-415ff46851dd\") " pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:40:46.366186 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:46.366150    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pmkzv\""
May 25 16:40:46.373393 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:46.373359    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l57q2"
May 25 16:40:46.495394 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:46.495362    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l57q2"]
May 25 16:40:46.498700 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:40:46.498673    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6646a861_65d0_4b6f_af0c_415ff46851dd.slice/crio-a033f0c0115494bcbe346c54cadf2d20edf58598624208b6b163194972c50e85 WatchSource:0}: Error finding container a033f0c0115494bcbe346c54cadf2d20edf58598624208b6b163194972c50e85: Status 404 returned error can't find the container with id a033f0c0115494bcbe346c54cadf2d20edf58598624208b6b163194972c50e85
May 25 16:40:47.162320 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:47.162271    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l57q2" event={"ID":"6646a861-65d0-4b6f-af0c-415ff46851dd","Type":"ContainerStarted","Data":"a033f0c0115494bcbe346c54cadf2d20edf58598624208b6b163194972c50e85"}
May 25 16:40:48.166374 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:48.166339    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l57q2" event={"ID":"6646a861-65d0-4b6f-af0c-415ff46851dd","Type":"ContainerStarted","Data":"63455628caef413f4f3a35b1dc01301d963dda621f7aa1643c6962e30cbfac37"}
May 25 16:40:48.166374 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:48.166380    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l57q2" event={"ID":"6646a861-65d0-4b6f-af0c-415ff46851dd","Type":"ContainerStarted","Data":"f129854dc1f426572590aed6921b87890effbdd1be4c55815f3c8f5a80290f1c"}
May 25 16:40:48.184551 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:40:48.184492    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l57q2" podStartSLOduration=252.034639366 podStartE2EDuration="4m13.184475572s" podCreationTimestamp="2026-05-25 16:36:35 +0000 UTC" firstStartedPulling="2026-05-25 16:40:46.500775541 +0000 UTC m=+251.848693770" lastFinishedPulling="2026-05-25 16:40:47.650611741 +0000 UTC m=+252.998529976" observedRunningTime="2026-05-25 16:40:48.182426006 +0000 UTC m=+253.530344257" watchObservedRunningTime="2026-05-25 16:40:48.184475572 +0000 UTC m=+253.532393822"
May 25 16:41:35.165780 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:41:35.165747    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 16:41:35.166835 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:41:35.166816    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 16:41:35.169069 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:41:35.169049    2554 kubelet.go:1628] "Image garbage collection succeeded"
May 25 16:44:00.786963 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:00.786857    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-z2pb2"]
May 25 16:44:00.787418 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:00.787132    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ba6020a-e04d-4c62-836f-058b291b1d4b" containerName="registry"
May 25 16:44:00.787418 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:00.787144    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ba6020a-e04d-4c62-836f-058b291b1d4b" containerName="registry"
May 25 16:44:00.787418 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:00.787195    2554 memory_manager.go:356] "RemoveStaleState removing state" podUID="5ba6020a-e04d-4c62-836f-058b291b1d4b" containerName="registry"
May 25 16:44:00.789855 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:00.789834    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-z2pb2"
May 25 16:44:00.792430 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:00.792405    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
May 25 16:44:00.793478 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:00.793454    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-zkfq9\""
May 25 16:44:00.793590 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:00.793484    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
May 25 16:44:00.797926 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:00.797898    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-z2pb2"]
May 25 16:44:00.953280 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:00.953232    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skgnh\" (UniqueName: \"kubernetes.io/projected/e0f5cb7d-9942-4dca-ab17-a3db8e23fcd2-kube-api-access-skgnh\") pod \"cert-manager-79c8d999ff-z2pb2\" (UID: \"e0f5cb7d-9942-4dca-ab17-a3db8e23fcd2\") " pod="cert-manager/cert-manager-79c8d999ff-z2pb2"
May 25 16:44:00.953483 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:00.953324    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e0f5cb7d-9942-4dca-ab17-a3db8e23fcd2-bound-sa-token\") pod \"cert-manager-79c8d999ff-z2pb2\" (UID: \"e0f5cb7d-9942-4dca-ab17-a3db8e23fcd2\") " pod="cert-manager/cert-manager-79c8d999ff-z2pb2"
May 25 16:44:01.054379 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:01.054278    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e0f5cb7d-9942-4dca-ab17-a3db8e23fcd2-bound-sa-token\") pod \"cert-manager-79c8d999ff-z2pb2\" (UID: \"e0f5cb7d-9942-4dca-ab17-a3db8e23fcd2\") " pod="cert-manager/cert-manager-79c8d999ff-z2pb2"
May 25 16:44:01.054379 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:01.054327    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-skgnh\" (UniqueName: \"kubernetes.io/projected/e0f5cb7d-9942-4dca-ab17-a3db8e23fcd2-kube-api-access-skgnh\") pod \"cert-manager-79c8d999ff-z2pb2\" (UID: \"e0f5cb7d-9942-4dca-ab17-a3db8e23fcd2\") " pod="cert-manager/cert-manager-79c8d999ff-z2pb2"
May 25 16:44:01.063534 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:01.063497    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e0f5cb7d-9942-4dca-ab17-a3db8e23fcd2-bound-sa-token\") pod \"cert-manager-79c8d999ff-z2pb2\" (UID: \"e0f5cb7d-9942-4dca-ab17-a3db8e23fcd2\") " pod="cert-manager/cert-manager-79c8d999ff-z2pb2"
May 25 16:44:01.063704 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:01.063684    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skgnh\" (UniqueName: \"kubernetes.io/projected/e0f5cb7d-9942-4dca-ab17-a3db8e23fcd2-kube-api-access-skgnh\") pod \"cert-manager-79c8d999ff-z2pb2\" (UID: \"e0f5cb7d-9942-4dca-ab17-a3db8e23fcd2\") " pod="cert-manager/cert-manager-79c8d999ff-z2pb2"
May 25 16:44:01.099715 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:01.099675    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-z2pb2"
May 25 16:44:01.232480 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:01.232446    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-z2pb2"]
May 25 16:44:01.235710 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:44:01.235677    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0f5cb7d_9942_4dca_ab17_a3db8e23fcd2.slice/crio-5bde5ce6c2089200bf686d14c8923a9cbe14f5af1d47b1cba48d046c0ba76f6e WatchSource:0}: Error finding container 5bde5ce6c2089200bf686d14c8923a9cbe14f5af1d47b1cba48d046c0ba76f6e: Status 404 returned error can't find the container with id 5bde5ce6c2089200bf686d14c8923a9cbe14f5af1d47b1cba48d046c0ba76f6e
May 25 16:44:01.237501 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:01.237482    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 16:44:01.648283 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:01.648245    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-z2pb2" event={"ID":"e0f5cb7d-9942-4dca-ab17-a3db8e23fcd2","Type":"ContainerStarted","Data":"5bde5ce6c2089200bf686d14c8923a9cbe14f5af1d47b1cba48d046c0ba76f6e"}
May 25 16:44:04.660084 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:04.660043    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-z2pb2" event={"ID":"e0f5cb7d-9942-4dca-ab17-a3db8e23fcd2","Type":"ContainerStarted","Data":"b1efb11afaee5ee1dbb20c236a79fcac82a3a0d71aa19c83d39e7daf17305880"}
May 25 16:44:04.677791 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:04.677742    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-z2pb2" podStartSLOduration=2.114522606 podStartE2EDuration="4.677725624s" podCreationTimestamp="2026-05-25 16:44:00 +0000 UTC" firstStartedPulling="2026-05-25 16:44:01.237640658 +0000 UTC m=+446.585558886" lastFinishedPulling="2026-05-25 16:44:03.800843675 +0000 UTC m=+449.148761904" observedRunningTime="2026-05-25 16:44:04.676593136 +0000 UTC m=+450.024511391" watchObservedRunningTime="2026-05-25 16:44:04.677725624 +0000 UTC m=+450.025643877"
May 25 16:44:16.212501 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.212462    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q"]
May 25 16:44:16.215671 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.215650    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q"
May 25 16:44:16.218207 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.218179    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
May 25 16:44:16.218338 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.218205    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cnhwk\""
May 25 16:44:16.218338 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.218303    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
May 25 16:44:16.218613 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.218598    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
May 25 16:44:16.218716 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.218698    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
May 25 16:44:16.240049 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.240018    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q"]
May 25 16:44:16.257092 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.257059    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/906d0423-c29f-4f77-bb1b-f284cf6b6c8f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86d64d44d9-tpm2q\" (UID: \"906d0423-c29f-4f77-bb1b-f284cf6b6c8f\") " pod="opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q"
May 25 16:44:16.257092 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.257092    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg9hk\" (UniqueName: \"kubernetes.io/projected/906d0423-c29f-4f77-bb1b-f284cf6b6c8f-kube-api-access-dg9hk\") pod \"opendatahub-operator-controller-manager-86d64d44d9-tpm2q\" (UID: \"906d0423-c29f-4f77-bb1b-f284cf6b6c8f\") " pod="opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q"
May 25 16:44:16.257285 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.257118    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/906d0423-c29f-4f77-bb1b-f284cf6b6c8f-webhook-cert\") pod \"opendatahub-operator-controller-manager-86d64d44d9-tpm2q\" (UID: \"906d0423-c29f-4f77-bb1b-f284cf6b6c8f\") " pod="opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q"
May 25 16:44:16.357646 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.357604    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/906d0423-c29f-4f77-bb1b-f284cf6b6c8f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86d64d44d9-tpm2q\" (UID: \"906d0423-c29f-4f77-bb1b-f284cf6b6c8f\") " pod="opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q"
May 25 16:44:16.357646 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.357648    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dg9hk\" (UniqueName: \"kubernetes.io/projected/906d0423-c29f-4f77-bb1b-f284cf6b6c8f-kube-api-access-dg9hk\") pod \"opendatahub-operator-controller-manager-86d64d44d9-tpm2q\" (UID: \"906d0423-c29f-4f77-bb1b-f284cf6b6c8f\") " pod="opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q"
May 25 16:44:16.357851 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.357677    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/906d0423-c29f-4f77-bb1b-f284cf6b6c8f-webhook-cert\") pod \"opendatahub-operator-controller-manager-86d64d44d9-tpm2q\" (UID: \"906d0423-c29f-4f77-bb1b-f284cf6b6c8f\") " pod="opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q"
May 25 16:44:16.360224 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.360194    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/906d0423-c29f-4f77-bb1b-f284cf6b6c8f-webhook-cert\") pod \"opendatahub-operator-controller-manager-86d64d44d9-tpm2q\" (UID: \"906d0423-c29f-4f77-bb1b-f284cf6b6c8f\") " pod="opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q"
May 25 16:44:16.360345 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.360253    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/906d0423-c29f-4f77-bb1b-f284cf6b6c8f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86d64d44d9-tpm2q\" (UID: \"906d0423-c29f-4f77-bb1b-f284cf6b6c8f\") " pod="opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q"
May 25 16:44:16.366849 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.366821    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg9hk\" (UniqueName: \"kubernetes.io/projected/906d0423-c29f-4f77-bb1b-f284cf6b6c8f-kube-api-access-dg9hk\") pod \"opendatahub-operator-controller-manager-86d64d44d9-tpm2q\" (UID: \"906d0423-c29f-4f77-bb1b-f284cf6b6c8f\") " pod="opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q"
May 25 16:44:16.525763 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.525731    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q"
May 25 16:44:16.658652 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.658618    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q"]
May 25 16:44:16.661632 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:44:16.661602    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod906d0423_c29f_4f77_bb1b_f284cf6b6c8f.slice/crio-902e10a9903adec7cd60a5cdc85128747d3d6ae8e1fe7f407b91b44821b2a2f0 WatchSource:0}: Error finding container 902e10a9903adec7cd60a5cdc85128747d3d6ae8e1fe7f407b91b44821b2a2f0: Status 404 returned error can't find the container with id 902e10a9903adec7cd60a5cdc85128747d3d6ae8e1fe7f407b91b44821b2a2f0
May 25 16:44:16.696565 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:16.696521    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q" event={"ID":"906d0423-c29f-4f77-bb1b-f284cf6b6c8f","Type":"ContainerStarted","Data":"902e10a9903adec7cd60a5cdc85128747d3d6ae8e1fe7f407b91b44821b2a2f0"}
May 25 16:44:19.706378 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:19.706335    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q" event={"ID":"906d0423-c29f-4f77-bb1b-f284cf6b6c8f","Type":"ContainerStarted","Data":"c5d4be8ca953d16399329cc107a85c43afb7efc766d4058191a68ffd41c3a0cf"}
May 25 16:44:19.706727 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:19.706494    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q"
May 25 16:44:19.729524 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:19.729465    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q" podStartSLOduration=1.2705600320000001 podStartE2EDuration="3.729443165s" podCreationTimestamp="2026-05-25 16:44:16 +0000 UTC" firstStartedPulling="2026-05-25 16:44:16.663396538 +0000 UTC m=+462.011314767" lastFinishedPulling="2026-05-25 16:44:19.122279667 +0000 UTC m=+464.470197900" observedRunningTime="2026-05-25 16:44:19.727566623 +0000 UTC m=+465.075484886" watchObservedRunningTime="2026-05-25 16:44:19.729443165 +0000 UTC m=+465.077361417"
May 25 16:44:30.711462 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:30.711432    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-86d64d44d9-tpm2q"
May 25 16:44:33.952061 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:33.952028    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-77d58df487-cwxfg"]
May 25 16:44:33.958188 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:33.958159    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77d58df487-cwxfg"
May 25 16:44:33.961983 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:33.961959    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
May 25 16:44:33.963113 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:33.963098    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
May 25 16:44:33.965041 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:33.965013    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
May 25 16:44:33.965526 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:33.965499    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
May 25 16:44:33.965780 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:33.965765    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-q6zlz\""
May 25 16:44:33.972179 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:33.972155    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77d58df487-cwxfg"]
May 25 16:44:33.986219 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:33.986191    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkqng\" (UniqueName: \"kubernetes.io/projected/b886e360-1d17-4718-9ae4-db58180afb80-kube-api-access-jkqng\") pod \"kube-auth-proxy-77d58df487-cwxfg\" (UID: \"b886e360-1d17-4718-9ae4-db58180afb80\") " pod="openshift-ingress/kube-auth-proxy-77d58df487-cwxfg"
May 25 16:44:33.986369 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:33.986247    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b886e360-1d17-4718-9ae4-db58180afb80-tls-certs\") pod \"kube-auth-proxy-77d58df487-cwxfg\" (UID: \"b886e360-1d17-4718-9ae4-db58180afb80\") " pod="openshift-ingress/kube-auth-proxy-77d58df487-cwxfg"
May 25 16:44:33.986369 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:33.986316    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b886e360-1d17-4718-9ae4-db58180afb80-tmp\") pod \"kube-auth-proxy-77d58df487-cwxfg\" (UID: \"b886e360-1d17-4718-9ae4-db58180afb80\") " pod="openshift-ingress/kube-auth-proxy-77d58df487-cwxfg"
May 25 16:44:34.086701 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:34.086655    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b886e360-1d17-4718-9ae4-db58180afb80-tls-certs\") pod \"kube-auth-proxy-77d58df487-cwxfg\" (UID: \"b886e360-1d17-4718-9ae4-db58180afb80\") " pod="openshift-ingress/kube-auth-proxy-77d58df487-cwxfg"
May 25 16:44:34.086701 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:34.086710    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b886e360-1d17-4718-9ae4-db58180afb80-tmp\") pod \"kube-auth-proxy-77d58df487-cwxfg\" (UID: \"b886e360-1d17-4718-9ae4-db58180afb80\") " pod="openshift-ingress/kube-auth-proxy-77d58df487-cwxfg"
May 25 16:44:34.086981 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:34.086733    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jkqng\" (UniqueName: \"kubernetes.io/projected/b886e360-1d17-4718-9ae4-db58180afb80-kube-api-access-jkqng\") pod \"kube-auth-proxy-77d58df487-cwxfg\" (UID: \"b886e360-1d17-4718-9ae4-db58180afb80\") " pod="openshift-ingress/kube-auth-proxy-77d58df487-cwxfg"
May 25 16:44:34.089107 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:34.089071    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b886e360-1d17-4718-9ae4-db58180afb80-tmp\") pod \"kube-auth-proxy-77d58df487-cwxfg\" (UID: \"b886e360-1d17-4718-9ae4-db58180afb80\") " pod="openshift-ingress/kube-auth-proxy-77d58df487-cwxfg"
May 25 16:44:34.089315 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:34.089286    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b886e360-1d17-4718-9ae4-db58180afb80-tls-certs\") pod \"kube-auth-proxy-77d58df487-cwxfg\" (UID: \"b886e360-1d17-4718-9ae4-db58180afb80\") " pod="openshift-ingress/kube-auth-proxy-77d58df487-cwxfg"
May 25 16:44:34.096254 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:34.096230    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkqng\" (UniqueName: \"kubernetes.io/projected/b886e360-1d17-4718-9ae4-db58180afb80-kube-api-access-jkqng\") pod \"kube-auth-proxy-77d58df487-cwxfg\" (UID: \"b886e360-1d17-4718-9ae4-db58180afb80\") " pod="openshift-ingress/kube-auth-proxy-77d58df487-cwxfg"
May 25 16:44:34.267662 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:34.267624    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77d58df487-cwxfg"
May 25 16:44:34.408843 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:34.408808    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77d58df487-cwxfg"]
May 25 16:44:34.411940 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:44:34.411910    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb886e360_1d17_4718_9ae4_db58180afb80.slice/crio-ee9f81911c5d16388d216b088748a0762f463713fb740ad7ad30407fa29e7275 WatchSource:0}: Error finding container ee9f81911c5d16388d216b088748a0762f463713fb740ad7ad30407fa29e7275: Status 404 returned error can't find the container with id ee9f81911c5d16388d216b088748a0762f463713fb740ad7ad30407fa29e7275
May 25 16:44:34.749388 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:34.749301    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77d58df487-cwxfg" event={"ID":"b886e360-1d17-4718-9ae4-db58180afb80","Type":"ContainerStarted","Data":"ee9f81911c5d16388d216b088748a0762f463713fb740ad7ad30407fa29e7275"}
May 25 16:44:37.090249 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:37.090220    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
May 25 16:44:37.453425 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:37.453335    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-6lt69"]
May 25 16:44:37.456603 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:37.456583    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-6lt69"
May 25 16:44:37.459177 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:37.459145    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
May 25 16:44:37.459311 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:37.459146    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-h4bfj\""
May 25 16:44:37.466603 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:37.466574    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-6lt69"]
May 25 16:44:37.514005 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:37.513970    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6e4679e-daaf-4547-971d-b0522e3dd7ec-cert\") pod \"odh-model-controller-858dbf95b8-6lt69\" (UID: \"b6e4679e-daaf-4547-971d-b0522e3dd7ec\") " pod="opendatahub/odh-model-controller-858dbf95b8-6lt69"
May 25 16:44:37.514180 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:37.514014    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs6zp\" (UniqueName: \"kubernetes.io/projected/b6e4679e-daaf-4547-971d-b0522e3dd7ec-kube-api-access-bs6zp\") pod \"odh-model-controller-858dbf95b8-6lt69\" (UID: \"b6e4679e-daaf-4547-971d-b0522e3dd7ec\") " pod="opendatahub/odh-model-controller-858dbf95b8-6lt69"
May 25 16:44:37.615216 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:37.615175    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6e4679e-daaf-4547-971d-b0522e3dd7ec-cert\") pod \"odh-model-controller-858dbf95b8-6lt69\" (UID: \"b6e4679e-daaf-4547-971d-b0522e3dd7ec\") " pod="opendatahub/odh-model-controller-858dbf95b8-6lt69"
May 25 16:44:37.615216 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:37.615224    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bs6zp\" (UniqueName: \"kubernetes.io/projected/b6e4679e-daaf-4547-971d-b0522e3dd7ec-kube-api-access-bs6zp\") pod \"odh-model-controller-858dbf95b8-6lt69\" (UID: \"b6e4679e-daaf-4547-971d-b0522e3dd7ec\") " pod="opendatahub/odh-model-controller-858dbf95b8-6lt69"
May 25 16:44:37.615469 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:44:37.615346    2554 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
May 25 16:44:37.615469 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:44:37.615427    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6e4679e-daaf-4547-971d-b0522e3dd7ec-cert podName:b6e4679e-daaf-4547-971d-b0522e3dd7ec nodeName:}" failed. No retries permitted until 2026-05-25 16:44:38.115406236 +0000 UTC m=+483.463324481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b6e4679e-daaf-4547-971d-b0522e3dd7ec-cert") pod "odh-model-controller-858dbf95b8-6lt69" (UID: "b6e4679e-daaf-4547-971d-b0522e3dd7ec") : secret "odh-model-controller-webhook-cert" not found
May 25 16:44:37.624386 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:37.624350    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs6zp\" (UniqueName: \"kubernetes.io/projected/b6e4679e-daaf-4547-971d-b0522e3dd7ec-kube-api-access-bs6zp\") pod \"odh-model-controller-858dbf95b8-6lt69\" (UID: \"b6e4679e-daaf-4547-971d-b0522e3dd7ec\") " pod="opendatahub/odh-model-controller-858dbf95b8-6lt69"
May 25 16:44:37.759089 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:37.759052    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77d58df487-cwxfg" event={"ID":"b886e360-1d17-4718-9ae4-db58180afb80","Type":"ContainerStarted","Data":"6c4a826fe866437f7a402490cd51d371e875c26de8d1f41b9851ea4fe7ec3aca"}
May 25 16:44:37.776104 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:37.776051    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-77d58df487-cwxfg" podStartSLOduration=2.102221137 podStartE2EDuration="4.776036574s" podCreationTimestamp="2026-05-25 16:44:33 +0000 UTC" firstStartedPulling="2026-05-25 16:44:34.413714166 +0000 UTC m=+479.761632399" lastFinishedPulling="2026-05-25 16:44:37.087529604 +0000 UTC m=+482.435447836" observedRunningTime="2026-05-25 16:44:37.774925697 +0000 UTC m=+483.122843949" watchObservedRunningTime="2026-05-25 16:44:37.776036574 +0000 UTC m=+483.123954825"
May 25 16:44:38.119945 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:38.119834    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6e4679e-daaf-4547-971d-b0522e3dd7ec-cert\") pod \"odh-model-controller-858dbf95b8-6lt69\" (UID: \"b6e4679e-daaf-4547-971d-b0522e3dd7ec\") " pod="opendatahub/odh-model-controller-858dbf95b8-6lt69"
May 25 16:44:38.122357 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:38.122329    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b6e4679e-daaf-4547-971d-b0522e3dd7ec-cert\") pod \"odh-model-controller-858dbf95b8-6lt69\" (UID: \"b6e4679e-daaf-4547-971d-b0522e3dd7ec\") " pod="opendatahub/odh-model-controller-858dbf95b8-6lt69"
May 25 16:44:38.367458 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:38.367411    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-6lt69"
May 25 16:44:38.495232 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:38.495200    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-6lt69"]
May 25 16:44:38.499061 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:44:38.499035    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e4679e_daaf_4547_971d_b0522e3dd7ec.slice/crio-3e84a15f88ac0c0c23758bb1e7a94f28cc23c9be581fccf43ee62a3c34a3bd2d WatchSource:0}: Error finding container 3e84a15f88ac0c0c23758bb1e7a94f28cc23c9be581fccf43ee62a3c34a3bd2d: Status 404 returned error can't find the container with id 3e84a15f88ac0c0c23758bb1e7a94f28cc23c9be581fccf43ee62a3c34a3bd2d
May 25 16:44:38.762977 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:38.762943    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-6lt69" event={"ID":"b6e4679e-daaf-4547-971d-b0522e3dd7ec","Type":"ContainerStarted","Data":"3e84a15f88ac0c0c23758bb1e7a94f28cc23c9be581fccf43ee62a3c34a3bd2d"}
May 25 16:44:41.773381 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:41.773343    2554 generic.go:358] "Generic (PLEG): container finished" podID="b6e4679e-daaf-4547-971d-b0522e3dd7ec" containerID="303b5c867d4d3eb69b77b1bef8f426ea66adaa47a8543b1e273ee471071a948f" exitCode=1
May 25 16:44:41.773836 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:41.773397    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-6lt69" event={"ID":"b6e4679e-daaf-4547-971d-b0522e3dd7ec","Type":"ContainerDied","Data":"303b5c867d4d3eb69b77b1bef8f426ea66adaa47a8543b1e273ee471071a948f"}
May 25 16:44:41.773836 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:41.773650    2554 scope.go:117] "RemoveContainer" containerID="303b5c867d4d3eb69b77b1bef8f426ea66adaa47a8543b1e273ee471071a948f"
May 25 16:44:42.778439 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:42.778401    2554 generic.go:358] "Generic (PLEG): container finished" podID="b6e4679e-daaf-4547-971d-b0522e3dd7ec" containerID="8e0db006a6e1dbe616bfffa3d292299720207bdfb67769ed0ddfc750d5c6cda7" exitCode=1
May 25 16:44:42.778900 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:42.778456    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-6lt69" event={"ID":"b6e4679e-daaf-4547-971d-b0522e3dd7ec","Type":"ContainerDied","Data":"8e0db006a6e1dbe616bfffa3d292299720207bdfb67769ed0ddfc750d5c6cda7"}
May 25 16:44:42.778900 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:42.778494    2554 scope.go:117] "RemoveContainer" containerID="303b5c867d4d3eb69b77b1bef8f426ea66adaa47a8543b1e273ee471071a948f"
May 25 16:44:42.778900 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:42.778739    2554 scope.go:117] "RemoveContainer" containerID="8e0db006a6e1dbe616bfffa3d292299720207bdfb67769ed0ddfc750d5c6cda7"
May 25 16:44:42.779051 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:44:42.778932    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-6lt69_opendatahub(b6e4679e-daaf-4547-971d-b0522e3dd7ec)\"" pod="opendatahub/odh-model-controller-858dbf95b8-6lt69" podUID="b6e4679e-daaf-4547-971d-b0522e3dd7ec"
May 25 16:44:43.435965 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:43.435925    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-9b78t"]
May 25 16:44:43.440365 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:43.440333    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-9b78t"
May 25 16:44:43.443337 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:43.443310    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
May 25 16:44:43.443659 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:43.443637    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-p6wtq\""
May 25 16:44:43.449058 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:43.449029    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-9b78t"]
May 25 16:44:43.566009 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:43.565964    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de84230f-ae26-4d43-9851-e9682f74daa8-cert\") pod \"kserve-controller-manager-856948b99f-9b78t\" (UID: \"de84230f-ae26-4d43-9851-e9682f74daa8\") " pod="opendatahub/kserve-controller-manager-856948b99f-9b78t"
May 25 16:44:43.566248 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:43.566094    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhvc8\" (UniqueName: \"kubernetes.io/projected/de84230f-ae26-4d43-9851-e9682f74daa8-kube-api-access-zhvc8\") pod \"kserve-controller-manager-856948b99f-9b78t\" (UID: \"de84230f-ae26-4d43-9851-e9682f74daa8\") " pod="opendatahub/kserve-controller-manager-856948b99f-9b78t"
May 25 16:44:43.667283 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:43.667244    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de84230f-ae26-4d43-9851-e9682f74daa8-cert\") pod \"kserve-controller-manager-856948b99f-9b78t\" (UID: \"de84230f-ae26-4d43-9851-e9682f74daa8\") " pod="opendatahub/kserve-controller-manager-856948b99f-9b78t"
May 25 16:44:43.667459 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:43.667307    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zhvc8\" (UniqueName: \"kubernetes.io/projected/de84230f-ae26-4d43-9851-e9682f74daa8-kube-api-access-zhvc8\") pod \"kserve-controller-manager-856948b99f-9b78t\" (UID: \"de84230f-ae26-4d43-9851-e9682f74daa8\") " pod="opendatahub/kserve-controller-manager-856948b99f-9b78t"
May 25 16:44:43.667459 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:44:43.667416    2554 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
May 25 16:44:43.667594 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:44:43.667496    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de84230f-ae26-4d43-9851-e9682f74daa8-cert podName:de84230f-ae26-4d43-9851-e9682f74daa8 nodeName:}" failed. No retries permitted until 2026-05-25 16:44:44.167473531 +0000 UTC m=+489.515391760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/de84230f-ae26-4d43-9851-e9682f74daa8-cert") pod "kserve-controller-manager-856948b99f-9b78t" (UID: "de84230f-ae26-4d43-9851-e9682f74daa8") : secret "kserve-webhook-server-cert" not found
May 25 16:44:43.679654 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:43.679618    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhvc8\" (UniqueName: \"kubernetes.io/projected/de84230f-ae26-4d43-9851-e9682f74daa8-kube-api-access-zhvc8\") pod \"kserve-controller-manager-856948b99f-9b78t\" (UID: \"de84230f-ae26-4d43-9851-e9682f74daa8\") " pod="opendatahub/kserve-controller-manager-856948b99f-9b78t"
May 25 16:44:43.783909 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:43.783879    2554 scope.go:117] "RemoveContainer" containerID="8e0db006a6e1dbe616bfffa3d292299720207bdfb67769ed0ddfc750d5c6cda7"
May 25 16:44:43.784260 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:44:43.784126    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-6lt69_opendatahub(b6e4679e-daaf-4547-971d-b0522e3dd7ec)\"" pod="opendatahub/odh-model-controller-858dbf95b8-6lt69" podUID="b6e4679e-daaf-4547-971d-b0522e3dd7ec"
May 25 16:44:44.172707 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:44.172609    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de84230f-ae26-4d43-9851-e9682f74daa8-cert\") pod \"kserve-controller-manager-856948b99f-9b78t\" (UID: \"de84230f-ae26-4d43-9851-e9682f74daa8\") " pod="opendatahub/kserve-controller-manager-856948b99f-9b78t"
May 25 16:44:44.175111 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:44.175085    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de84230f-ae26-4d43-9851-e9682f74daa8-cert\") pod \"kserve-controller-manager-856948b99f-9b78t\" (UID: \"de84230f-ae26-4d43-9851-e9682f74daa8\") " pod="opendatahub/kserve-controller-manager-856948b99f-9b78t"
May 25 16:44:44.353338 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:44.353290    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-9b78t"
May 25 16:44:44.481588 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:44.481556    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-9b78t"]
May 25 16:44:44.484661 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:44:44.484629    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde84230f_ae26_4d43_9851_e9682f74daa8.slice/crio-d2134800bd38f26dceef1006edf82293c4134c41dd2ab53fc48f7e4e5e45279c WatchSource:0}: Error finding container d2134800bd38f26dceef1006edf82293c4134c41dd2ab53fc48f7e4e5e45279c: Status 404 returned error can't find the container with id d2134800bd38f26dceef1006edf82293c4134c41dd2ab53fc48f7e4e5e45279c
May 25 16:44:44.787771 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:44.787735    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-9b78t" event={"ID":"de84230f-ae26-4d43-9851-e9682f74daa8","Type":"ContainerStarted","Data":"d2134800bd38f26dceef1006edf82293c4134c41dd2ab53fc48f7e4e5e45279c"}
May 25 16:44:47.797986 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:47.797948    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-9b78t" event={"ID":"de84230f-ae26-4d43-9851-e9682f74daa8","Type":"ContainerStarted","Data":"8824dc04b66fa3fd269d1711301d328728b1736b5e196f9be0a69f4a520057f2"}
May 25 16:44:47.798369 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:47.798052    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-856948b99f-9b78t"
May 25 16:44:47.846734 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:47.846677    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-856948b99f-9b78t" podStartSLOduration=2.262407025 podStartE2EDuration="4.846659917s" podCreationTimestamp="2026-05-25 16:44:43 +0000 UTC" firstStartedPulling="2026-05-25 16:44:44.48604971 +0000 UTC m=+489.833967939" lastFinishedPulling="2026-05-25 16:44:47.070302601 +0000 UTC m=+492.418220831" observedRunningTime="2026-05-25 16:44:47.846047855 +0000 UTC m=+493.193966106" watchObservedRunningTime="2026-05-25 16:44:47.846659917 +0000 UTC m=+493.194578168"
May 25 16:44:48.368074 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:48.368036    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-6lt69"
May 25 16:44:48.368502 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:48.368487    2554 scope.go:117] "RemoveContainer" containerID="8e0db006a6e1dbe616bfffa3d292299720207bdfb67769ed0ddfc750d5c6cda7"
May 25 16:44:48.368669 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:44:48.368649    2554 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-6lt69_opendatahub(b6e4679e-daaf-4547-971d-b0522e3dd7ec)\"" pod="opendatahub/odh-model-controller-858dbf95b8-6lt69" podUID="b6e4679e-daaf-4547-971d-b0522e3dd7ec"
May 25 16:44:49.160272 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:49.160233    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-tr8g6"]
May 25 16:44:49.208762 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:49.208721    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-tr8g6"]
May 25 16:44:49.208962 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:49.208891    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-tr8g6"
May 25 16:44:49.211839 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:49.211815    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\""
May 25 16:44:49.211972 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:49.211839    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\""
May 25 16:44:49.212117 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:49.212103    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-hr4p6\""
May 25 16:44:49.311178 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:49.311133    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67mnp\" (UniqueName: \"kubernetes.io/projected/c171a954-84b5-4b75-9fda-29bd7b4ec06b-kube-api-access-67mnp\") pod \"servicemesh-operator3-55f49c5f94-tr8g6\" (UID: \"c171a954-84b5-4b75-9fda-29bd7b4ec06b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-tr8g6"
May 25 16:44:49.311361 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:49.311208    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/c171a954-84b5-4b75-9fda-29bd7b4ec06b-operator-config\") pod \"servicemesh-operator3-55f49c5f94-tr8g6\" (UID: \"c171a954-84b5-4b75-9fda-29bd7b4ec06b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-tr8g6"
May 25 16:44:49.412269 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:49.412171    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-67mnp\" (UniqueName: \"kubernetes.io/projected/c171a954-84b5-4b75-9fda-29bd7b4ec06b-kube-api-access-67mnp\") pod \"servicemesh-operator3-55f49c5f94-tr8g6\" (UID: \"c171a954-84b5-4b75-9fda-29bd7b4ec06b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-tr8g6"
May 25 16:44:49.412269 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:49.412218    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/c171a954-84b5-4b75-9fda-29bd7b4ec06b-operator-config\") pod \"servicemesh-operator3-55f49c5f94-tr8g6\" (UID: \"c171a954-84b5-4b75-9fda-29bd7b4ec06b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-tr8g6"
May 25 16:44:49.414848 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:49.414823    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/c171a954-84b5-4b75-9fda-29bd7b4ec06b-operator-config\") pod \"servicemesh-operator3-55f49c5f94-tr8g6\" (UID: \"c171a954-84b5-4b75-9fda-29bd7b4ec06b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-tr8g6"
May 25 16:44:49.427898 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:49.427838    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67mnp\" (UniqueName: \"kubernetes.io/projected/c171a954-84b5-4b75-9fda-29bd7b4ec06b-kube-api-access-67mnp\") pod \"servicemesh-operator3-55f49c5f94-tr8g6\" (UID: \"c171a954-84b5-4b75-9fda-29bd7b4ec06b\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-tr8g6"
May 25 16:44:49.517849 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:49.517808    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-tr8g6"
May 25 16:44:49.658594 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:49.658553    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-tr8g6"]
May 25 16:44:49.663528 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:44:49.663459    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc171a954_84b5_4b75_9fda_29bd7b4ec06b.slice/crio-dfe76baf0e1eb1f6cda8eda32a8f2044f3cbd67546d5656034dfd106eeb4df8b WatchSource:0}: Error finding container dfe76baf0e1eb1f6cda8eda32a8f2044f3cbd67546d5656034dfd106eeb4df8b: Status 404 returned error can't find the container with id dfe76baf0e1eb1f6cda8eda32a8f2044f3cbd67546d5656034dfd106eeb4df8b
May 25 16:44:49.805101 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:49.805063    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-tr8g6" event={"ID":"c171a954-84b5-4b75-9fda-29bd7b4ec06b","Type":"ContainerStarted","Data":"dfe76baf0e1eb1f6cda8eda32a8f2044f3cbd67546d5656034dfd106eeb4df8b"}
May 25 16:44:52.816675 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:52.816621    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-tr8g6" event={"ID":"c171a954-84b5-4b75-9fda-29bd7b4ec06b","Type":"ContainerStarted","Data":"2853a693a9078cdc67b9020e34064e33a80e6d70f52d11e319004b6b0fb3df57"}
May 25 16:44:52.817189 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:52.816764    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-tr8g6"
May 25 16:44:52.842196 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:52.842129    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-tr8g6" podStartSLOduration=1.7075366349999999 podStartE2EDuration="3.842108996s" podCreationTimestamp="2026-05-25 16:44:49 +0000 UTC" firstStartedPulling="2026-05-25 16:44:49.667114836 +0000 UTC m=+495.015033065" lastFinishedPulling="2026-05-25 16:44:51.801687194 +0000 UTC m=+497.149605426" observedRunningTime="2026-05-25 16:44:52.839442295 +0000 UTC m=+498.187360547" watchObservedRunningTime="2026-05-25 16:44:52.842108996 +0000 UTC m=+498.190027248"
May 25 16:44:58.368145 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:58.368102    2554 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-858dbf95b8-6lt69"
May 25 16:44:58.368534 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:58.368479    2554 scope.go:117] "RemoveContainer" containerID="8e0db006a6e1dbe616bfffa3d292299720207bdfb67769ed0ddfc750d5c6cda7"
May 25 16:44:58.839378 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:58.839342    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-6lt69" event={"ID":"b6e4679e-daaf-4547-971d-b0522e3dd7ec","Type":"ContainerStarted","Data":"602f652d6460f54f3fd4347c5da4b14b858b76cb8ffeed4f9db498c19dbe0a26"}
May 25 16:44:58.839585 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:58.839565    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-6lt69"
May 25 16:44:58.858759 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:44:58.858710    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-858dbf95b8-6lt69" podStartSLOduration=1.7169901300000001 podStartE2EDuration="21.858694716s" podCreationTimestamp="2026-05-25 16:44:37 +0000 UTC" firstStartedPulling="2026-05-25 16:44:38.500379177 +0000 UTC m=+483.848297406" lastFinishedPulling="2026-05-25 16:44:58.64208376 +0000 UTC m=+503.990001992" observedRunningTime="2026-05-25 16:44:58.858270514 +0000 UTC m=+504.206188765" watchObservedRunningTime="2026-05-25 16:44:58.858694716 +0000 UTC m=+504.206612967"
May 25 16:45:03.821804 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:03.821771    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-tr8g6"
May 25 16:45:04.954935 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:04.954891    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"]
May 25 16:45:04.958139 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:04.958113    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:04.962124 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:04.962097    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
May 25 16:45:04.962280 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:04.962196    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
May 25 16:45:04.962654 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:04.962633    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
May 25 16:45:04.962858 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:04.962837    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fr569\""
May 25 16:45:04.963298 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:04.963266    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
May 25 16:45:04.972916 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:04.972847    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"]
May 25 16:45:05.042637 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.042600    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3c165474-1178-4dc1-a2b8-29b7958369b0-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.042637 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.042644    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3c165474-1178-4dc1-a2b8-29b7958369b0-cacerts\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.042928 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.042664    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgqg6\" (UniqueName: \"kubernetes.io/projected/3c165474-1178-4dc1-a2b8-29b7958369b0-kube-api-access-xgqg6\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.042928 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.042763    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3c165474-1178-4dc1-a2b8-29b7958369b0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.042928 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.042807    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c165474-1178-4dc1-a2b8-29b7958369b0-local-certs\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.042928 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.042900    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3c165474-1178-4dc1-a2b8-29b7958369b0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.043083 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.042929    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c165474-1178-4dc1-a2b8-29b7958369b0-istio-token\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.143346 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.143300    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3c165474-1178-4dc1-a2b8-29b7958369b0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.143508 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.143354    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c165474-1178-4dc1-a2b8-29b7958369b0-local-certs\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.143508 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.143402    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3c165474-1178-4dc1-a2b8-29b7958369b0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.143508 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.143428    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c165474-1178-4dc1-a2b8-29b7958369b0-istio-token\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.143508 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.143474    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3c165474-1178-4dc1-a2b8-29b7958369b0-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.143508 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.143509    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3c165474-1178-4dc1-a2b8-29b7958369b0-cacerts\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.143791 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.143531    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xgqg6\" (UniqueName: \"kubernetes.io/projected/3c165474-1178-4dc1-a2b8-29b7958369b0-kube-api-access-xgqg6\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.144087 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.144060    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3c165474-1178-4dc1-a2b8-29b7958369b0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.146263 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.146231    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c165474-1178-4dc1-a2b8-29b7958369b0-local-certs\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.146356 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.146236    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3c165474-1178-4dc1-a2b8-29b7958369b0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.146477 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.146453    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3c165474-1178-4dc1-a2b8-29b7958369b0-cacerts\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.146731 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.146712    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3c165474-1178-4dc1-a2b8-29b7958369b0-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.159018 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.158984    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgqg6\" (UniqueName: \"kubernetes.io/projected/3c165474-1178-4dc1-a2b8-29b7958369b0-kube-api-access-xgqg6\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.159018 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.159001    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c165474-1178-4dc1-a2b8-29b7958369b0-istio-token\") pod \"istiod-openshift-gateway-798958bb55-qhfrf\" (UID: \"3c165474-1178-4dc1-a2b8-29b7958369b0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.269089 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.269061    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:05.428174 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.428141    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"]
May 25 16:45:05.432367 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:45:05.432334    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c165474_1178_4dc1_a2b8_29b7958369b0.slice/crio-f640f312704ceb644b607e2fce9b34e2c3ce6b7b867ddb3c9342fce9414c5741 WatchSource:0}: Error finding container f640f312704ceb644b607e2fce9b34e2c3ce6b7b867ddb3c9342fce9414c5741: Status 404 returned error can't find the container with id f640f312704ceb644b607e2fce9b34e2c3ce6b7b867ddb3c9342fce9414c5741
May 25 16:45:05.862327 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:05.862289    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf" event={"ID":"3c165474-1178-4dc1-a2b8-29b7958369b0","Type":"ContainerStarted","Data":"f640f312704ceb644b607e2fce9b34e2c3ce6b7b867ddb3c9342fce9414c5741"}
May 25 16:45:08.152830 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:08.152787    2554 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
May 25 16:45:08.153153 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:08.152859    2554 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
May 25 16:45:08.874009 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:08.873969    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf" event={"ID":"3c165474-1178-4dc1-a2b8-29b7958369b0","Type":"ContainerStarted","Data":"6e7a95a72b6fd6a4edc037559b85c6749c87f176d97495afb05022a3c1b44e2c"}
May 25 16:45:08.874190 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:08.874090    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:08.894843 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:08.894783    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf" podStartSLOduration=2.177162334 podStartE2EDuration="4.894767489s" podCreationTimestamp="2026-05-25 16:45:04 +0000 UTC" firstStartedPulling="2026-05-25 16:45:05.434935984 +0000 UTC m=+510.782854219" lastFinishedPulling="2026-05-25 16:45:08.152541128 +0000 UTC m=+513.500459374" observedRunningTime="2026-05-25 16:45:08.894376418 +0000 UTC m=+514.242294673" watchObservedRunningTime="2026-05-25 16:45:08.894767489 +0000 UTC m=+514.242685741"
May 25 16:45:09.846122 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:09.846092    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-858dbf95b8-6lt69"
May 25 16:45:09.880578 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:09.880547    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-qhfrf"
May 25 16:45:18.806586 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:45:18.806555    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-856948b99f-9b78t"
May 25 16:46:10.411594 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:10.411556    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-xk7bb"]
May 25 16:46:10.416418 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:10.416397    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-xk7bb"
May 25 16:46:10.419335 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:10.419314    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jg8ks\""
May 25 16:46:10.419460 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:10.419450    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
May 25 16:46:10.420481 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:10.420466    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
May 25 16:46:10.420533 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:10.420475    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
May 25 16:46:10.429466 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:10.429442    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-xk7bb"]
May 25 16:46:10.555255 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:10.555218    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k66rt\" (UniqueName: \"kubernetes.io/projected/66ae27e7-5f35-40eb-9596-796730721c49-kube-api-access-k66rt\") pod \"dns-operator-controller-manager-648d5c98bc-xk7bb\" (UID: \"66ae27e7-5f35-40eb-9596-796730721c49\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-xk7bb"
May 25 16:46:10.655728 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:10.655690    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k66rt\" (UniqueName: \"kubernetes.io/projected/66ae27e7-5f35-40eb-9596-796730721c49-kube-api-access-k66rt\") pod \"dns-operator-controller-manager-648d5c98bc-xk7bb\" (UID: \"66ae27e7-5f35-40eb-9596-796730721c49\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-xk7bb"
May 25 16:46:10.668300 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:10.668228    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k66rt\" (UniqueName: \"kubernetes.io/projected/66ae27e7-5f35-40eb-9596-796730721c49-kube-api-access-k66rt\") pod \"dns-operator-controller-manager-648d5c98bc-xk7bb\" (UID: \"66ae27e7-5f35-40eb-9596-796730721c49\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-xk7bb"
May 25 16:46:10.725944 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:10.725900    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-xk7bb"
May 25 16:46:10.855256 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:10.855218    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-xk7bb"]
May 25 16:46:10.857829 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:46:10.857798    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66ae27e7_5f35_40eb_9596_796730721c49.slice/crio-1d7396c5b57e063a45a22106bcf6a19287afcae100a2e4bcc63c493f952e14f0 WatchSource:0}: Error finding container 1d7396c5b57e063a45a22106bcf6a19287afcae100a2e4bcc63c493f952e14f0: Status 404 returned error can't find the container with id 1d7396c5b57e063a45a22106bcf6a19287afcae100a2e4bcc63c493f952e14f0
May 25 16:46:11.071963 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:11.071909    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-xk7bb" event={"ID":"66ae27e7-5f35-40eb-9596-796730721c49","Type":"ContainerStarted","Data":"1d7396c5b57e063a45a22106bcf6a19287afcae100a2e4bcc63c493f952e14f0"}
May 25 16:46:13.083196 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:13.083010    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-xk7bb" event={"ID":"66ae27e7-5f35-40eb-9596-796730721c49","Type":"ContainerStarted","Data":"44996657b19760f85c8fcc6b05c81b3b8a743431ec25424eb66997292de968c2"}
May 25 16:46:13.083196 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:13.083158    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-xk7bb"
May 25 16:46:13.111775 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:13.111715    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-xk7bb" podStartSLOduration=1.157944131 podStartE2EDuration="3.111697271s" podCreationTimestamp="2026-05-25 16:46:10 +0000 UTC" firstStartedPulling="2026-05-25 16:46:10.859817217 +0000 UTC m=+576.207735445" lastFinishedPulling="2026-05-25 16:46:12.813570357 +0000 UTC m=+578.161488585" observedRunningTime="2026-05-25 16:46:13.109408647 +0000 UTC m=+578.457326901" watchObservedRunningTime="2026-05-25 16:46:13.111697271 +0000 UTC m=+578.459615523"
May 25 16:46:13.119798 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:13.119763    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6"]
May 25 16:46:13.123045 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:13.123017    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6"
May 25 16:46:13.125777 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:13.125742    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6tdms\""
May 25 16:46:13.132982 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:13.132944    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6"]
May 25 16:46:13.280517 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:13.280473    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45fh2\" (UniqueName: \"kubernetes.io/projected/b1be452a-8380-4822-be28-2e1fae49a58b-kube-api-access-45fh2\") pod \"limitador-operator-controller-manager-85c4996f8c-p6ls6\" (UID: \"b1be452a-8380-4822-be28-2e1fae49a58b\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6"
May 25 16:46:13.381944 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:13.381829    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-45fh2\" (UniqueName: \"kubernetes.io/projected/b1be452a-8380-4822-be28-2e1fae49a58b-kube-api-access-45fh2\") pod \"limitador-operator-controller-manager-85c4996f8c-p6ls6\" (UID: \"b1be452a-8380-4822-be28-2e1fae49a58b\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6"
May 25 16:46:13.390971 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:13.390934    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45fh2\" (UniqueName: \"kubernetes.io/projected/b1be452a-8380-4822-be28-2e1fae49a58b-kube-api-access-45fh2\") pod \"limitador-operator-controller-manager-85c4996f8c-p6ls6\" (UID: \"b1be452a-8380-4822-be28-2e1fae49a58b\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6"
May 25 16:46:13.436558 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:13.436515    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6"
May 25 16:46:13.591351 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:13.591250    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6"]
May 25 16:46:13.594137 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:46:13.594108    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1be452a_8380_4822_be28_2e1fae49a58b.slice/crio-2e0c7128c8c0de4b3c88fdeb7ab7741e524b7b0c091fd7bc363e4722adb56f87 WatchSource:0}: Error finding container 2e0c7128c8c0de4b3c88fdeb7ab7741e524b7b0c091fd7bc363e4722adb56f87: Status 404 returned error can't find the container with id 2e0c7128c8c0de4b3c88fdeb7ab7741e524b7b0c091fd7bc363e4722adb56f87
May 25 16:46:14.088642 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:14.088604    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6" event={"ID":"b1be452a-8380-4822-be28-2e1fae49a58b","Type":"ContainerStarted","Data":"2e0c7128c8c0de4b3c88fdeb7ab7741e524b7b0c091fd7bc363e4722adb56f87"}
May 25 16:46:16.095822 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:16.095782    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6" event={"ID":"b1be452a-8380-4822-be28-2e1fae49a58b","Type":"ContainerStarted","Data":"351ea854a0e02f36cfb9e64260c5a31f8b191e468b0420e7b79449a1af1946a6"}
May 25 16:46:16.096248 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:16.095836    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6"
May 25 16:46:16.113019 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:16.112964    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6" podStartSLOduration=1.207712744 podStartE2EDuration="3.112949086s" podCreationTimestamp="2026-05-25 16:46:13 +0000 UTC" firstStartedPulling="2026-05-25 16:46:13.595955111 +0000 UTC m=+578.943873339" lastFinishedPulling="2026-05-25 16:46:15.50119145 +0000 UTC m=+580.849109681" observedRunningTime="2026-05-25 16:46:16.111619283 +0000 UTC m=+581.459537533" watchObservedRunningTime="2026-05-25 16:46:16.112949086 +0000 UTC m=+581.460867393"
May 25 16:46:24.090888 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:24.090829    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-xk7bb"
May 25 16:46:27.101835 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:27.101804    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6"
May 25 16:46:35.191265 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:35.191232    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 16:46:35.191741 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:35.191413    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 16:46:36.369614 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.369578    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6"]
May 25 16:46:36.370003 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.369842    2554 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6" podUID="b1be452a-8380-4822-be28-2e1fae49a58b" containerName="manager" containerID="cri-o://351ea854a0e02f36cfb9e64260c5a31f8b191e468b0420e7b79449a1af1946a6" gracePeriod=2
May 25 16:46:36.384604 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.384569    2554 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6"]
May 25 16:46:36.392208 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.392175    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-w89xq"]
May 25 16:46:36.392493 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.392479    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1be452a-8380-4822-be28-2e1fae49a58b" containerName="manager"
May 25 16:46:36.392493 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.392494    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1be452a-8380-4822-be28-2e1fae49a58b" containerName="manager"
May 25 16:46:36.392611 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.392547    2554 memory_manager.go:356] "RemoveStaleState removing state" podUID="b1be452a-8380-4822-be28-2e1fae49a58b" containerName="manager"
May 25 16:46:36.396125 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.396105    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-w89xq"
May 25 16:46:36.410472 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.410439    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-w89xq"]
May 25 16:46:36.572626 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.572581    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8btj\" (UniqueName: \"kubernetes.io/projected/b5fbd335-caf1-4bee-bcd3-ff3abedc59cd-kube-api-access-d8btj\") pod \"limitador-operator-controller-manager-85c4996f8c-w89xq\" (UID: \"b5fbd335-caf1-4bee-bcd3-ff3abedc59cd\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-w89xq"
May 25 16:46:36.610674 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.610650    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6"
May 25 16:46:36.613156 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.613130    2554 status_manager.go:895] "Failed to get status for pod" podUID="b1be452a-8380-4822-be28-2e1fae49a58b" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6" err="pods \"limitador-operator-controller-manager-85c4996f8c-p6ls6\" is forbidden: User \"system:node:ip-10-0-135-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-137.ec2.internal' and this object"
May 25 16:46:36.674169 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.674081    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d8btj\" (UniqueName: \"kubernetes.io/projected/b5fbd335-caf1-4bee-bcd3-ff3abedc59cd-kube-api-access-d8btj\") pod \"limitador-operator-controller-manager-85c4996f8c-w89xq\" (UID: \"b5fbd335-caf1-4bee-bcd3-ff3abedc59cd\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-w89xq"
May 25 16:46:36.685800 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.685767    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8btj\" (UniqueName: \"kubernetes.io/projected/b5fbd335-caf1-4bee-bcd3-ff3abedc59cd-kube-api-access-d8btj\") pod \"limitador-operator-controller-manager-85c4996f8c-w89xq\" (UID: \"b5fbd335-caf1-4bee-bcd3-ff3abedc59cd\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-w89xq"
May 25 16:46:36.757227 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.757192    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-w89xq"
May 25 16:46:36.775112 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.775075    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45fh2\" (UniqueName: \"kubernetes.io/projected/b1be452a-8380-4822-be28-2e1fae49a58b-kube-api-access-45fh2\") pod \"b1be452a-8380-4822-be28-2e1fae49a58b\" (UID: \"b1be452a-8380-4822-be28-2e1fae49a58b\") "
May 25 16:46:36.777314 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.777271    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1be452a-8380-4822-be28-2e1fae49a58b-kube-api-access-45fh2" (OuterVolumeSpecName: "kube-api-access-45fh2") pod "b1be452a-8380-4822-be28-2e1fae49a58b" (UID: "b1be452a-8380-4822-be28-2e1fae49a58b"). InnerVolumeSpecName "kube-api-access-45fh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 16:46:36.875844 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.875813    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45fh2\" (UniqueName: \"kubernetes.io/projected/b1be452a-8380-4822-be28-2e1fae49a58b-kube-api-access-45fh2\") on node \"ip-10-0-135-137.ec2.internal\" DevicePath \"\""
May 25 16:46:36.894948 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:36.894910    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-w89xq"]
May 25 16:46:36.901632 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:46:36.901602    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5fbd335_caf1_4bee_bcd3_ff3abedc59cd.slice/crio-e38917aa0e8a8313c8ad6781cf1168c68bdee1b5b5106cd17258dde61729f892 WatchSource:0}: Error finding container e38917aa0e8a8313c8ad6781cf1168c68bdee1b5b5106cd17258dde61729f892: Status 404 returned error can't find the container with id e38917aa0e8a8313c8ad6781cf1168c68bdee1b5b5106cd17258dde61729f892
May 25 16:46:37.170779 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:37.170745    2554 generic.go:358] "Generic (PLEG): container finished" podID="b1be452a-8380-4822-be28-2e1fae49a58b" containerID="351ea854a0e02f36cfb9e64260c5a31f8b191e468b0420e7b79449a1af1946a6" exitCode=0
May 25 16:46:37.170967 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:37.170795    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6"
May 25 16:46:37.170967 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:37.170853    2554 scope.go:117] "RemoveContainer" containerID="351ea854a0e02f36cfb9e64260c5a31f8b191e468b0420e7b79449a1af1946a6"
May 25 16:46:37.172236 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:37.172210    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-w89xq" event={"ID":"b5fbd335-caf1-4bee-bcd3-ff3abedc59cd","Type":"ContainerStarted","Data":"4b2aac501be6c8f4dd367bde97982087ddb5da97ed2016f0aa6d876955d6ff22"}
May 25 16:46:37.172346 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:37.172247    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-w89xq" event={"ID":"b5fbd335-caf1-4bee-bcd3-ff3abedc59cd","Type":"ContainerStarted","Data":"e38917aa0e8a8313c8ad6781cf1168c68bdee1b5b5106cd17258dde61729f892"}
May 25 16:46:37.172346 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:37.172313    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-w89xq"
May 25 16:46:37.173500 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:37.173476    2554 status_manager.go:895] "Failed to get status for pod" podUID="b1be452a-8380-4822-be28-2e1fae49a58b" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6" err="pods \"limitador-operator-controller-manager-85c4996f8c-p6ls6\" is forbidden: User \"system:node:ip-10-0-135-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-137.ec2.internal' and this object"
May 25 16:46:37.178949 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:37.178930    2554 scope.go:117] "RemoveContainer" containerID="351ea854a0e02f36cfb9e64260c5a31f8b191e468b0420e7b79449a1af1946a6"
May 25 16:46:37.179257 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:46:37.179233    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"351ea854a0e02f36cfb9e64260c5a31f8b191e468b0420e7b79449a1af1946a6\": container with ID starting with 351ea854a0e02f36cfb9e64260c5a31f8b191e468b0420e7b79449a1af1946a6 not found: ID does not exist" containerID="351ea854a0e02f36cfb9e64260c5a31f8b191e468b0420e7b79449a1af1946a6"
May 25 16:46:37.179308 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:37.179268    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"351ea854a0e02f36cfb9e64260c5a31f8b191e468b0420e7b79449a1af1946a6"} err="failed to get container status \"351ea854a0e02f36cfb9e64260c5a31f8b191e468b0420e7b79449a1af1946a6\": rpc error: code = NotFound desc = could not find container \"351ea854a0e02f36cfb9e64260c5a31f8b191e468b0420e7b79449a1af1946a6\": container with ID starting with 351ea854a0e02f36cfb9e64260c5a31f8b191e468b0420e7b79449a1af1946a6 not found: ID does not exist"
May 25 16:46:37.200521 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:37.200442    2554 status_manager.go:895] "Failed to get status for pod" podUID="b1be452a-8380-4822-be28-2e1fae49a58b" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6" err="pods \"limitador-operator-controller-manager-85c4996f8c-p6ls6\" is forbidden: User \"system:node:ip-10-0-135-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-137.ec2.internal' and this object"
May 25 16:46:37.202548 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:37.202512    2554 status_manager.go:895] "Failed to get status for pod" podUID="b1be452a-8380-4822-be28-2e1fae49a58b" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-p6ls6" err="pods \"limitador-operator-controller-manager-85c4996f8c-p6ls6\" is forbidden: User \"system:node:ip-10-0-135-137.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-137.ec2.internal' and this object"
May 25 16:46:37.267119 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:37.267084    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1be452a-8380-4822-be28-2e1fae49a58b" path="/var/lib/kubelet/pods/b1be452a-8380-4822-be28-2e1fae49a58b/volumes"
May 25 16:46:48.178411 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:48.178383    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-w89xq"
May 25 16:46:48.199809 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:46:48.199756    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-w89xq" podStartSLOduration=12.199740563 podStartE2EDuration="12.199740563s" podCreationTimestamp="2026-05-25 16:46:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 16:46:37.198337235 +0000 UTC m=+602.546255488" watchObservedRunningTime="2026-05-25 16:46:48.199740563 +0000 UTC m=+613.547658811"
May 25 16:47:20.836735 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:20.836697    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-87fw5"]
May 25 16:47:20.839983 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:20.839963    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5"
May 25 16:47:20.842616 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:20.842591    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-28zv7\""
May 25 16:47:20.842616 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:20.842591    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
May 25 16:47:20.847725 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:20.847702    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-87fw5"]
May 25 16:47:20.924654 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:20.924619    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/71ed4d64-b1f8-451b-ab57-d52305d46560-config-file\") pod \"limitador-limitador-7d549b5b-87fw5\" (UID: \"71ed4d64-b1f8-451b-ab57-d52305d46560\") " pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5"
May 25 16:47:20.924806 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:20.924771    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8qcf\" (UniqueName: \"kubernetes.io/projected/71ed4d64-b1f8-451b-ab57-d52305d46560-kube-api-access-p8qcf\") pod \"limitador-limitador-7d549b5b-87fw5\" (UID: \"71ed4d64-b1f8-451b-ab57-d52305d46560\") " pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5"
May 25 16:47:20.934616 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:20.934586    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-87fw5"]
May 25 16:47:21.025387 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:21.025329    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p8qcf\" (UniqueName: \"kubernetes.io/projected/71ed4d64-b1f8-451b-ab57-d52305d46560-kube-api-access-p8qcf\") pod \"limitador-limitador-7d549b5b-87fw5\" (UID: \"71ed4d64-b1f8-451b-ab57-d52305d46560\") " pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5"
May 25 16:47:21.025592 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:21.025397    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/71ed4d64-b1f8-451b-ab57-d52305d46560-config-file\") pod \"limitador-limitador-7d549b5b-87fw5\" (UID: \"71ed4d64-b1f8-451b-ab57-d52305d46560\") " pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5"
May 25 16:47:21.026070 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:21.026051    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/71ed4d64-b1f8-451b-ab57-d52305d46560-config-file\") pod \"limitador-limitador-7d549b5b-87fw5\" (UID: \"71ed4d64-b1f8-451b-ab57-d52305d46560\") " pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5"
May 25 16:47:21.036261 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:21.036234    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8qcf\" (UniqueName: \"kubernetes.io/projected/71ed4d64-b1f8-451b-ab57-d52305d46560-kube-api-access-p8qcf\") pod \"limitador-limitador-7d549b5b-87fw5\" (UID: \"71ed4d64-b1f8-451b-ab57-d52305d46560\") " pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5"
May 25 16:47:21.151320 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:21.151231    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5"
May 25 16:47:21.303956 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:21.303932    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-87fw5"]
May 25 16:47:21.306724 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:47:21.306690    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ed4d64_b1f8_451b_ab57_d52305d46560.slice/crio-94903b9be83a29d27d904671655c070143bb2253478e9ba8adde450b4a273f41 WatchSource:0}: Error finding container 94903b9be83a29d27d904671655c070143bb2253478e9ba8adde450b4a273f41: Status 404 returned error can't find the container with id 94903b9be83a29d27d904671655c070143bb2253478e9ba8adde450b4a273f41
May 25 16:47:21.314639 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:21.314613    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5" event={"ID":"71ed4d64-b1f8-451b-ab57-d52305d46560","Type":"ContainerStarted","Data":"94903b9be83a29d27d904671655c070143bb2253478e9ba8adde450b4a273f41"}
May 25 16:47:24.328957 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:24.328917    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5" event={"ID":"71ed4d64-b1f8-451b-ab57-d52305d46560","Type":"ContainerStarted","Data":"4f274bd2754ce16b615f2c0bda187328f42171364160e17eeac1aac78fd5b494"}
May 25 16:47:24.329411 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:24.329041    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5"
May 25 16:47:24.347648 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:24.347590    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5" podStartSLOduration=1.787006052 podStartE2EDuration="4.347572733s" podCreationTimestamp="2026-05-25 16:47:20 +0000 UTC" firstStartedPulling="2026-05-25 16:47:21.308501032 +0000 UTC m=+646.656419260" lastFinishedPulling="2026-05-25 16:47:23.86906771 +0000 UTC m=+649.216985941" observedRunningTime="2026-05-25 16:47:24.345293675 +0000 UTC m=+649.693211934" watchObservedRunningTime="2026-05-25 16:47:24.347572733 +0000 UTC m=+649.695491026"
May 25 16:47:35.332967 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:35.332934    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5"
May 25 16:47:35.647831 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:35.647790    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-87fw5"]
May 25 16:47:35.648093 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:35.648048    2554 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5" podUID="71ed4d64-b1f8-451b-ab57-d52305d46560" containerName="limitador" containerID="cri-o://4f274bd2754ce16b615f2c0bda187328f42171364160e17eeac1aac78fd5b494" gracePeriod=30
May 25 16:47:36.205611 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:36.205583    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5"
May 25 16:47:36.354704 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:36.354666    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/71ed4d64-b1f8-451b-ab57-d52305d46560-config-file\") pod \"71ed4d64-b1f8-451b-ab57-d52305d46560\" (UID: \"71ed4d64-b1f8-451b-ab57-d52305d46560\") "
May 25 16:47:36.355091 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:36.354725    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8qcf\" (UniqueName: \"kubernetes.io/projected/71ed4d64-b1f8-451b-ab57-d52305d46560-kube-api-access-p8qcf\") pod \"71ed4d64-b1f8-451b-ab57-d52305d46560\" (UID: \"71ed4d64-b1f8-451b-ab57-d52305d46560\") "
May 25 16:47:36.355149 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:36.355104    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71ed4d64-b1f8-451b-ab57-d52305d46560-config-file" (OuterVolumeSpecName: "config-file") pod "71ed4d64-b1f8-451b-ab57-d52305d46560" (UID: "71ed4d64-b1f8-451b-ab57-d52305d46560"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 16:47:36.356785 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:36.356762    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71ed4d64-b1f8-451b-ab57-d52305d46560-kube-api-access-p8qcf" (OuterVolumeSpecName: "kube-api-access-p8qcf") pod "71ed4d64-b1f8-451b-ab57-d52305d46560" (UID: "71ed4d64-b1f8-451b-ab57-d52305d46560"). InnerVolumeSpecName "kube-api-access-p8qcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 16:47:36.368391 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:36.368361    2554 generic.go:358] "Generic (PLEG): container finished" podID="71ed4d64-b1f8-451b-ab57-d52305d46560" containerID="4f274bd2754ce16b615f2c0bda187328f42171364160e17eeac1aac78fd5b494" exitCode=0
May 25 16:47:36.368544 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:36.368419    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5"
May 25 16:47:36.368544 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:36.368435    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5" event={"ID":"71ed4d64-b1f8-451b-ab57-d52305d46560","Type":"ContainerDied","Data":"4f274bd2754ce16b615f2c0bda187328f42171364160e17eeac1aac78fd5b494"}
May 25 16:47:36.368544 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:36.368460    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-87fw5" event={"ID":"71ed4d64-b1f8-451b-ab57-d52305d46560","Type":"ContainerDied","Data":"94903b9be83a29d27d904671655c070143bb2253478e9ba8adde450b4a273f41"}
May 25 16:47:36.368544 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:36.368473    2554 scope.go:117] "RemoveContainer" containerID="4f274bd2754ce16b615f2c0bda187328f42171364160e17eeac1aac78fd5b494"
May 25 16:47:36.376584 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:36.376565    2554 scope.go:117] "RemoveContainer" containerID="4f274bd2754ce16b615f2c0bda187328f42171364160e17eeac1aac78fd5b494"
May 25 16:47:36.376877 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:47:36.376836    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f274bd2754ce16b615f2c0bda187328f42171364160e17eeac1aac78fd5b494\": container with ID starting with 4f274bd2754ce16b615f2c0bda187328f42171364160e17eeac1aac78fd5b494 not found: ID does not exist" containerID="4f274bd2754ce16b615f2c0bda187328f42171364160e17eeac1aac78fd5b494"
May 25 16:47:36.376983 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:36.376895    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f274bd2754ce16b615f2c0bda187328f42171364160e17eeac1aac78fd5b494"} err="failed to get container status \"4f274bd2754ce16b615f2c0bda187328f42171364160e17eeac1aac78fd5b494\": rpc error: code = NotFound desc = could not find container \"4f274bd2754ce16b615f2c0bda187328f42171364160e17eeac1aac78fd5b494\": container with ID starting with 4f274bd2754ce16b615f2c0bda187328f42171364160e17eeac1aac78fd5b494 not found: ID does not exist"
May 25 16:47:36.390061 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:36.390028    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-87fw5"]
May 25 16:47:36.393547 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:36.393524    2554 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-87fw5"]
May 25 16:47:36.456201 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:36.456164    2554 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/71ed4d64-b1f8-451b-ab57-d52305d46560-config-file\") on node \"ip-10-0-135-137.ec2.internal\" DevicePath \"\""
May 25 16:47:36.456201 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:36.456194    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8qcf\" (UniqueName: \"kubernetes.io/projected/71ed4d64-b1f8-451b-ab57-d52305d46560-kube-api-access-p8qcf\") on node \"ip-10-0-135-137.ec2.internal\" DevicePath \"\""
May 25 16:47:37.266332 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:37.266300    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71ed4d64-b1f8-451b-ab57-d52305d46560" path="/var/lib/kubelet/pods/71ed4d64-b1f8-451b-ab57-d52305d46560/volumes"
May 25 16:47:41.416486 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:41.416451    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-wmpqx"]
May 25 16:47:41.417007 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:41.416984    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71ed4d64-b1f8-451b-ab57-d52305d46560" containerName="limitador"
May 25 16:47:41.417105 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:41.417008    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="71ed4d64-b1f8-451b-ab57-d52305d46560" containerName="limitador"
May 25 16:47:41.417105 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:41.417098    2554 memory_manager.go:356] "RemoveStaleState removing state" podUID="71ed4d64-b1f8-451b-ab57-d52305d46560" containerName="limitador"
May 25 16:47:41.421557 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:41.421532    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-wmpqx"
May 25 16:47:41.424183 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:41.424157    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
May 25 16:47:41.424317 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:41.424246    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tgbf2\""
May 25 16:47:41.430656 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:41.430625    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-wmpqx"]
May 25 16:47:41.594944 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:41.594909    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpnvb\" (UniqueName: \"kubernetes.io/projected/cb4ccb90-d2c9-437d-b38c-b6496ab33948-kube-api-access-xpnvb\") pod \"postgres-68c4fbbd6f-wmpqx\" (UID: \"cb4ccb90-d2c9-437d-b38c-b6496ab33948\") " pod="opendatahub/postgres-68c4fbbd6f-wmpqx"
May 25 16:47:41.595120 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:41.594967    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/cb4ccb90-d2c9-437d-b38c-b6496ab33948-data\") pod \"postgres-68c4fbbd6f-wmpqx\" (UID: \"cb4ccb90-d2c9-437d-b38c-b6496ab33948\") " pod="opendatahub/postgres-68c4fbbd6f-wmpqx"
May 25 16:47:41.695596 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:41.695513    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xpnvb\" (UniqueName: \"kubernetes.io/projected/cb4ccb90-d2c9-437d-b38c-b6496ab33948-kube-api-access-xpnvb\") pod \"postgres-68c4fbbd6f-wmpqx\" (UID: \"cb4ccb90-d2c9-437d-b38c-b6496ab33948\") " pod="opendatahub/postgres-68c4fbbd6f-wmpqx"
May 25 16:47:41.695596 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:41.695562    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/cb4ccb90-d2c9-437d-b38c-b6496ab33948-data\") pod \"postgres-68c4fbbd6f-wmpqx\" (UID: \"cb4ccb90-d2c9-437d-b38c-b6496ab33948\") " pod="opendatahub/postgres-68c4fbbd6f-wmpqx"
May 25 16:47:41.695985 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:41.695963    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/cb4ccb90-d2c9-437d-b38c-b6496ab33948-data\") pod \"postgres-68c4fbbd6f-wmpqx\" (UID: \"cb4ccb90-d2c9-437d-b38c-b6496ab33948\") " pod="opendatahub/postgres-68c4fbbd6f-wmpqx"
May 25 16:47:41.704174 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:41.704152    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpnvb\" (UniqueName: \"kubernetes.io/projected/cb4ccb90-d2c9-437d-b38c-b6496ab33948-kube-api-access-xpnvb\") pod \"postgres-68c4fbbd6f-wmpqx\" (UID: \"cb4ccb90-d2c9-437d-b38c-b6496ab33948\") " pod="opendatahub/postgres-68c4fbbd6f-wmpqx"
May 25 16:47:41.734147 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:41.734104    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-wmpqx"
May 25 16:47:41.862351 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:41.862317    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-wmpqx"]
May 25 16:47:41.865846 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:47:41.865819    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb4ccb90_d2c9_437d_b38c_b6496ab33948.slice/crio-49c0a256a8931b0c397aeb756b20bdebacf96b6410343690569ed8e3cc2749a0 WatchSource:0}: Error finding container 49c0a256a8931b0c397aeb756b20bdebacf96b6410343690569ed8e3cc2749a0: Status 404 returned error can't find the container with id 49c0a256a8931b0c397aeb756b20bdebacf96b6410343690569ed8e3cc2749a0
May 25 16:47:42.388181 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:42.388145    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-wmpqx" event={"ID":"cb4ccb90-d2c9-437d-b38c-b6496ab33948","Type":"ContainerStarted","Data":"49c0a256a8931b0c397aeb756b20bdebacf96b6410343690569ed8e3cc2749a0"}
May 25 16:47:47.408186 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:47.408146    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-wmpqx" event={"ID":"cb4ccb90-d2c9-437d-b38c-b6496ab33948","Type":"ContainerStarted","Data":"f00be0412ee15555b1e1d1bfddc9efec4de09a83d39d490bbfc86230707f2f94"}
May 25 16:47:47.408627 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:47.408208    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-wmpqx"
May 25 16:47:47.423710 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:47.423655    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-wmpqx" podStartSLOduration=0.970146517 podStartE2EDuration="6.423642607s" podCreationTimestamp="2026-05-25 16:47:41 +0000 UTC" firstStartedPulling="2026-05-25 16:47:41.867007448 +0000 UTC m=+667.214925676" lastFinishedPulling="2026-05-25 16:47:47.320503525 +0000 UTC m=+672.668421766" observedRunningTime="2026-05-25 16:47:47.422838257 +0000 UTC m=+672.770756520" watchObservedRunningTime="2026-05-25 16:47:47.423642607 +0000 UTC m=+672.771560857"
May 25 16:47:53.439772 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:53.439741    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-wmpqx"
May 25 16:47:59.136240 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.136203    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d8bc65dcf-xlfm7"]
May 25 16:47:59.249260 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.249226    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d8bc65dcf-xlfm7"]
May 25 16:47:59.249453 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.249373    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8bc65dcf-xlfm7"
May 25 16:47:59.252463 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.252442    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
May 25 16:47:59.252575 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.252442    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-sdnrv\""
May 25 16:47:59.294342 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.294306    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b757cc664-sqc4q"]
May 25 16:47:59.308692 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.308662    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b757cc664-sqc4q"]
May 25 16:47:59.308899 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.308805    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b757cc664-sqc4q"
May 25 16:47:59.335199 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.335145    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwjps\" (UniqueName: \"kubernetes.io/projected/521866a1-f3b4-41bc-8c38-01c7d8b13e81-kube-api-access-vwjps\") pod \"maas-controller-7d8bc65dcf-xlfm7\" (UID: \"521866a1-f3b4-41bc-8c38-01c7d8b13e81\") " pod="opendatahub/maas-controller-7d8bc65dcf-xlfm7"
May 25 16:47:59.335370 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.335254    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dhvj\" (UniqueName: \"kubernetes.io/projected/81859385-a343-4abb-96e8-6450a18b17e3-kube-api-access-8dhvj\") pod \"maas-controller-7b757cc664-sqc4q\" (UID: \"81859385-a343-4abb-96e8-6450a18b17e3\") " pod="opendatahub/maas-controller-7b757cc664-sqc4q"
May 25 16:47:59.435901 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.435789    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhvj\" (UniqueName: \"kubernetes.io/projected/81859385-a343-4abb-96e8-6450a18b17e3-kube-api-access-8dhvj\") pod \"maas-controller-7b757cc664-sqc4q\" (UID: \"81859385-a343-4abb-96e8-6450a18b17e3\") " pod="opendatahub/maas-controller-7b757cc664-sqc4q"
May 25 16:47:59.435901 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.435838    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vwjps\" (UniqueName: \"kubernetes.io/projected/521866a1-f3b4-41bc-8c38-01c7d8b13e81-kube-api-access-vwjps\") pod \"maas-controller-7d8bc65dcf-xlfm7\" (UID: \"521866a1-f3b4-41bc-8c38-01c7d8b13e81\") " pod="opendatahub/maas-controller-7d8bc65dcf-xlfm7"
May 25 16:47:59.447295 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.447259    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwjps\" (UniqueName: \"kubernetes.io/projected/521866a1-f3b4-41bc-8c38-01c7d8b13e81-kube-api-access-vwjps\") pod \"maas-controller-7d8bc65dcf-xlfm7\" (UID: \"521866a1-f3b4-41bc-8c38-01c7d8b13e81\") " pod="opendatahub/maas-controller-7d8bc65dcf-xlfm7"
May 25 16:47:59.447434 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.447308    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dhvj\" (UniqueName: \"kubernetes.io/projected/81859385-a343-4abb-96e8-6450a18b17e3-kube-api-access-8dhvj\") pod \"maas-controller-7b757cc664-sqc4q\" (UID: \"81859385-a343-4abb-96e8-6450a18b17e3\") " pod="opendatahub/maas-controller-7b757cc664-sqc4q"
May 25 16:47:59.559949 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.559908    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8bc65dcf-xlfm7"
May 25 16:47:59.620232 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.620002    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b757cc664-sqc4q"
May 25 16:47:59.701826 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.701748    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d8bc65dcf-xlfm7"]
May 25 16:47:59.705542 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:47:59.705511    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod521866a1_f3b4_41bc_8c38_01c7d8b13e81.slice/crio-c87e6bc54fe1aa0837338bf16b404a365231c5121eb173811b2b39ca1ba2f103 WatchSource:0}: Error finding container c87e6bc54fe1aa0837338bf16b404a365231c5121eb173811b2b39ca1ba2f103: Status 404 returned error can't find the container with id c87e6bc54fe1aa0837338bf16b404a365231c5121eb173811b2b39ca1ba2f103
May 25 16:47:59.762414 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:47:59.762298    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b757cc664-sqc4q"]
May 25 16:47:59.764967 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:47:59.764937    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81859385_a343_4abb_96e8_6450a18b17e3.slice/crio-57f97ad16b446b5ea4630ef6c90210804880f7f1b8f0ec675ab24f898f143b79 WatchSource:0}: Error finding container 57f97ad16b446b5ea4630ef6c90210804880f7f1b8f0ec675ab24f898f143b79: Status 404 returned error can't find the container with id 57f97ad16b446b5ea4630ef6c90210804880f7f1b8f0ec675ab24f898f143b79
May 25 16:48:00.451315 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:00.451276    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b757cc664-sqc4q" event={"ID":"81859385-a343-4abb-96e8-6450a18b17e3","Type":"ContainerStarted","Data":"57f97ad16b446b5ea4630ef6c90210804880f7f1b8f0ec675ab24f898f143b79"}
May 25 16:48:00.453374 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:00.453325    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8bc65dcf-xlfm7" event={"ID":"521866a1-f3b4-41bc-8c38-01c7d8b13e81","Type":"ContainerStarted","Data":"c87e6bc54fe1aa0837338bf16b404a365231c5121eb173811b2b39ca1ba2f103"}
May 25 16:48:03.464782 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:03.464741    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b757cc664-sqc4q" event={"ID":"81859385-a343-4abb-96e8-6450a18b17e3","Type":"ContainerStarted","Data":"33e5cb39f3f3a2ccbe97648df47ba13ddd34970efe19e5056cf921dfcfe2967c"}
May 25 16:48:03.465272 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:03.464823    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b757cc664-sqc4q"
May 25 16:48:03.466129 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:03.466093    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8bc65dcf-xlfm7" event={"ID":"521866a1-f3b4-41bc-8c38-01c7d8b13e81","Type":"ContainerStarted","Data":"1cdebba1fdd31b75908db3f73d3d6ee7548f5ee8ab8c0daaf424eb00fc2a5f39"}
May 25 16:48:03.466233 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:03.466208    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d8bc65dcf-xlfm7"
May 25 16:48:03.483375 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:03.483324    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-7b757cc664-sqc4q" podStartSLOduration=1.777561363 podStartE2EDuration="4.483308141s" podCreationTimestamp="2026-05-25 16:47:59 +0000 UTC" firstStartedPulling="2026-05-25 16:47:59.766360222 +0000 UTC m=+685.114278451" lastFinishedPulling="2026-05-25 16:48:02.472106988 +0000 UTC m=+687.820025229" observedRunningTime="2026-05-25 16:48:03.481894588 +0000 UTC m=+688.829812840" watchObservedRunningTime="2026-05-25 16:48:03.483308141 +0000 UTC m=+688.831226452"
May 25 16:48:03.498777 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:03.498720    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-7d8bc65dcf-xlfm7" podStartSLOduration=1.723681453 podStartE2EDuration="4.498704652s" podCreationTimestamp="2026-05-25 16:47:59 +0000 UTC" firstStartedPulling="2026-05-25 16:47:59.706908334 +0000 UTC m=+685.054826563" lastFinishedPulling="2026-05-25 16:48:02.481931514 +0000 UTC m=+687.829849762" observedRunningTime="2026-05-25 16:48:03.497652395 +0000 UTC m=+688.845570648" watchObservedRunningTime="2026-05-25 16:48:03.498704652 +0000 UTC m=+688.846622902"
May 25 16:48:14.475085 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:14.475052    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b757cc664-sqc4q"
May 25 16:48:14.475462 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:14.475226    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d8bc65dcf-xlfm7"
May 25 16:48:14.544642 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:14.544577    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d8bc65dcf-xlfm7"]
May 25 16:48:14.544860 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:14.544786    2554 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-7d8bc65dcf-xlfm7" podUID="521866a1-f3b4-41bc-8c38-01c7d8b13e81" containerName="manager" containerID="cri-o://1cdebba1fdd31b75908db3f73d3d6ee7548f5ee8ab8c0daaf424eb00fc2a5f39" gracePeriod=30
May 25 16:48:14.789152 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:14.789126    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8bc65dcf-xlfm7"
May 25 16:48:14.866238 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:14.866193    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwjps\" (UniqueName: \"kubernetes.io/projected/521866a1-f3b4-41bc-8c38-01c7d8b13e81-kube-api-access-vwjps\") pod \"521866a1-f3b4-41bc-8c38-01c7d8b13e81\" (UID: \"521866a1-f3b4-41bc-8c38-01c7d8b13e81\") "
May 25 16:48:14.868405 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:14.868379    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/521866a1-f3b4-41bc-8c38-01c7d8b13e81-kube-api-access-vwjps" (OuterVolumeSpecName: "kube-api-access-vwjps") pod "521866a1-f3b4-41bc-8c38-01c7d8b13e81" (UID: "521866a1-f3b4-41bc-8c38-01c7d8b13e81"). InnerVolumeSpecName "kube-api-access-vwjps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 16:48:14.967120 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:14.967087    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwjps\" (UniqueName: \"kubernetes.io/projected/521866a1-f3b4-41bc-8c38-01c7d8b13e81-kube-api-access-vwjps\") on node \"ip-10-0-135-137.ec2.internal\" DevicePath \"\""
May 25 16:48:15.502261 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:15.502223    2554 generic.go:358] "Generic (PLEG): container finished" podID="521866a1-f3b4-41bc-8c38-01c7d8b13e81" containerID="1cdebba1fdd31b75908db3f73d3d6ee7548f5ee8ab8c0daaf424eb00fc2a5f39" exitCode=0
May 25 16:48:15.502707 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:15.502280    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8bc65dcf-xlfm7"
May 25 16:48:15.502707 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:15.502306    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8bc65dcf-xlfm7" event={"ID":"521866a1-f3b4-41bc-8c38-01c7d8b13e81","Type":"ContainerDied","Data":"1cdebba1fdd31b75908db3f73d3d6ee7548f5ee8ab8c0daaf424eb00fc2a5f39"}
May 25 16:48:15.502707 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:15.502343    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8bc65dcf-xlfm7" event={"ID":"521866a1-f3b4-41bc-8c38-01c7d8b13e81","Type":"ContainerDied","Data":"c87e6bc54fe1aa0837338bf16b404a365231c5121eb173811b2b39ca1ba2f103"}
May 25 16:48:15.502707 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:15.502358    2554 scope.go:117] "RemoveContainer" containerID="1cdebba1fdd31b75908db3f73d3d6ee7548f5ee8ab8c0daaf424eb00fc2a5f39"
May 25 16:48:15.510344 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:15.510326    2554 scope.go:117] "RemoveContainer" containerID="1cdebba1fdd31b75908db3f73d3d6ee7548f5ee8ab8c0daaf424eb00fc2a5f39"
May 25 16:48:15.510606 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:48:15.510577    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cdebba1fdd31b75908db3f73d3d6ee7548f5ee8ab8c0daaf424eb00fc2a5f39\": container with ID starting with 1cdebba1fdd31b75908db3f73d3d6ee7548f5ee8ab8c0daaf424eb00fc2a5f39 not found: ID does not exist" containerID="1cdebba1fdd31b75908db3f73d3d6ee7548f5ee8ab8c0daaf424eb00fc2a5f39"
May 25 16:48:15.510650 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:15.510614    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cdebba1fdd31b75908db3f73d3d6ee7548f5ee8ab8c0daaf424eb00fc2a5f39"} err="failed to get container status \"1cdebba1fdd31b75908db3f73d3d6ee7548f5ee8ab8c0daaf424eb00fc2a5f39\": rpc error: code = NotFound desc = could not find container \"1cdebba1fdd31b75908db3f73d3d6ee7548f5ee8ab8c0daaf424eb00fc2a5f39\": container with ID starting with 1cdebba1fdd31b75908db3f73d3d6ee7548f5ee8ab8c0daaf424eb00fc2a5f39 not found: ID does not exist"
May 25 16:48:15.519848 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:15.519819    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d8bc65dcf-xlfm7"]
May 25 16:48:15.524333 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:15.524273    2554 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d8bc65dcf-xlfm7"]
May 25 16:48:17.266728 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:17.266693    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="521866a1-f3b4-41bc-8c38-01c7d8b13e81" path="/var/lib/kubelet/pods/521866a1-f3b4-41bc-8c38-01c7d8b13e81/volumes"
May 25 16:48:31.321434 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.321329    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-bb4f4487f-kr6cm"]
May 25 16:48:31.321913 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.321655    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="521866a1-f3b4-41bc-8c38-01c7d8b13e81" containerName="manager"
May 25 16:48:31.321913 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.321668    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="521866a1-f3b4-41bc-8c38-01c7d8b13e81" containerName="manager"
May 25 16:48:31.321913 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.321733    2554 memory_manager.go:356] "RemoveStaleState removing state" podUID="521866a1-f3b4-41bc-8c38-01c7d8b13e81" containerName="manager"
May 25 16:48:31.324507 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.324489    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-bb4f4487f-kr6cm"
May 25 16:48:31.327302 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.327279    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
May 25 16:48:31.327431 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.327303    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-qtlbn\""
May 25 16:48:31.336348 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.336324    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-bb4f4487f-kr6cm"]
May 25 16:48:31.404456 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.404416    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd2325eb-4604-4461-bbd4-29b885a3272a-maas-api-tls\") pod \"maas-api-bb4f4487f-kr6cm\" (UID: \"cd2325eb-4604-4461-bbd4-29b885a3272a\") " pod="opendatahub/maas-api-bb4f4487f-kr6cm"
May 25 16:48:31.404633 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.404478    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26rv8\" (UniqueName: \"kubernetes.io/projected/cd2325eb-4604-4461-bbd4-29b885a3272a-kube-api-access-26rv8\") pod \"maas-api-bb4f4487f-kr6cm\" (UID: \"cd2325eb-4604-4461-bbd4-29b885a3272a\") " pod="opendatahub/maas-api-bb4f4487f-kr6cm"
May 25 16:48:31.504933 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.504848    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd2325eb-4604-4461-bbd4-29b885a3272a-maas-api-tls\") pod \"maas-api-bb4f4487f-kr6cm\" (UID: \"cd2325eb-4604-4461-bbd4-29b885a3272a\") " pod="opendatahub/maas-api-bb4f4487f-kr6cm"
May 25 16:48:31.505118 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.504956    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-26rv8\" (UniqueName: \"kubernetes.io/projected/cd2325eb-4604-4461-bbd4-29b885a3272a-kube-api-access-26rv8\") pod \"maas-api-bb4f4487f-kr6cm\" (UID: \"cd2325eb-4604-4461-bbd4-29b885a3272a\") " pod="opendatahub/maas-api-bb4f4487f-kr6cm"
May 25 16:48:31.507659 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.507630    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd2325eb-4604-4461-bbd4-29b885a3272a-maas-api-tls\") pod \"maas-api-bb4f4487f-kr6cm\" (UID: \"cd2325eb-4604-4461-bbd4-29b885a3272a\") " pod="opendatahub/maas-api-bb4f4487f-kr6cm"
May 25 16:48:31.514729 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.514695    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26rv8\" (UniqueName: \"kubernetes.io/projected/cd2325eb-4604-4461-bbd4-29b885a3272a-kube-api-access-26rv8\") pod \"maas-api-bb4f4487f-kr6cm\" (UID: \"cd2325eb-4604-4461-bbd4-29b885a3272a\") " pod="opendatahub/maas-api-bb4f4487f-kr6cm"
May 25 16:48:31.636216 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.636129    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-bb4f4487f-kr6cm"
May 25 16:48:31.722439 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.722403    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-6qnq8"]
May 25 16:48:31.727888 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.727813    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-6qnq8"
May 25 16:48:31.731688 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.731665    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
May 25 16:48:31.731812 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.731702    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-pdkfs\""
May 25 16:48:31.738562 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.738514    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-6qnq8"]
May 25 16:48:31.783916 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.783862    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-bb4f4487f-kr6cm"]
May 25 16:48:31.788267 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:48:31.788237    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2325eb_4604_4461_bbd4_29b885a3272a.slice/crio-89a6f047d35048e64b38260fc4537f3ca18bf9baab9bafb422484a853cc3d37c WatchSource:0}: Error finding container 89a6f047d35048e64b38260fc4537f3ca18bf9baab9bafb422484a853cc3d37c: Status 404 returned error can't find the container with id 89a6f047d35048e64b38260fc4537f3ca18bf9baab9bafb422484a853cc3d37c
May 25 16:48:31.808032 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.807998    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jccbt\" (UniqueName: \"kubernetes.io/projected/eaa49daf-ccbd-42a0-9b73-1c4bc8ea883f-kube-api-access-jccbt\") pod \"payload-processing-58b6c8fdc7-6qnq8\" (UID: \"eaa49daf-ccbd-42a0-9b73-1c4bc8ea883f\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-6qnq8"
May 25 16:48:31.909213 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.909115    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jccbt\" (UniqueName: \"kubernetes.io/projected/eaa49daf-ccbd-42a0-9b73-1c4bc8ea883f-kube-api-access-jccbt\") pod \"payload-processing-58b6c8fdc7-6qnq8\" (UID: \"eaa49daf-ccbd-42a0-9b73-1c4bc8ea883f\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-6qnq8"
May 25 16:48:31.923670 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:31.923643    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jccbt\" (UniqueName: \"kubernetes.io/projected/eaa49daf-ccbd-42a0-9b73-1c4bc8ea883f-kube-api-access-jccbt\") pod \"payload-processing-58b6c8fdc7-6qnq8\" (UID: \"eaa49daf-ccbd-42a0-9b73-1c4bc8ea883f\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-6qnq8"
May 25 16:48:32.042341 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:32.042307    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-6qnq8"
May 25 16:48:32.172765 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:32.172688    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-6qnq8"]
May 25 16:48:32.175877 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:48:32.175820    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaa49daf_ccbd_42a0_9b73_1c4bc8ea883f.slice/crio-85946b2db9358fde5356bd80878b134e9bab126580a9ecc9dfeff934a2f2d6cd WatchSource:0}: Error finding container 85946b2db9358fde5356bd80878b134e9bab126580a9ecc9dfeff934a2f2d6cd: Status 404 returned error can't find the container with id 85946b2db9358fde5356bd80878b134e9bab126580a9ecc9dfeff934a2f2d6cd
May 25 16:48:32.561895 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:32.561848    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-6qnq8" event={"ID":"eaa49daf-ccbd-42a0-9b73-1c4bc8ea883f","Type":"ContainerStarted","Data":"85946b2db9358fde5356bd80878b134e9bab126580a9ecc9dfeff934a2f2d6cd"}
May 25 16:48:32.563469 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:32.563438    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-bb4f4487f-kr6cm" event={"ID":"cd2325eb-4604-4461-bbd4-29b885a3272a","Type":"ContainerStarted","Data":"89a6f047d35048e64b38260fc4537f3ca18bf9baab9bafb422484a853cc3d37c"}
May 25 16:48:33.568721 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:33.568678    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-bb4f4487f-kr6cm" event={"ID":"cd2325eb-4604-4461-bbd4-29b885a3272a","Type":"ContainerStarted","Data":"4e826df1159de501c8e0d1cc3b6bfb5ed3f23f3a9571276c008d9eda4a896042"}
May 25 16:48:33.568721 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:33.568730    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-bb4f4487f-kr6cm"
May 25 16:48:33.570081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:33.570055    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-6qnq8" event={"ID":"eaa49daf-ccbd-42a0-9b73-1c4bc8ea883f","Type":"ContainerStarted","Data":"5c68e00df51fbe056ad186a065191df8225bebb5cd75cb8b02387527e1ea5cb7"}
May 25 16:48:33.570206 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:33.570187    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-6qnq8"
May 25 16:48:33.598083 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:33.598024    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-bb4f4487f-kr6cm" podStartSLOduration=1.348520288 podStartE2EDuration="2.598006771s" podCreationTimestamp="2026-05-25 16:48:31 +0000 UTC" firstStartedPulling="2026-05-25 16:48:31.789681549 +0000 UTC m=+717.137599778" lastFinishedPulling="2026-05-25 16:48:33.039168019 +0000 UTC m=+718.387086261" observedRunningTime="2026-05-25 16:48:33.595851764 +0000 UTC m=+718.943770013" watchObservedRunningTime="2026-05-25 16:48:33.598006771 +0000 UTC m=+718.945925022"
May 25 16:48:33.625052 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:33.625002    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58b6c8fdc7-6qnq8" podStartSLOduration=1.348549456 podStartE2EDuration="2.624985641s" podCreationTimestamp="2026-05-25 16:48:31 +0000 UTC" firstStartedPulling="2026-05-25 16:48:32.177675134 +0000 UTC m=+717.525593363" lastFinishedPulling="2026-05-25 16:48:33.454111304 +0000 UTC m=+718.802029548" observedRunningTime="2026-05-25 16:48:33.623369786 +0000 UTC m=+718.971288037" watchObservedRunningTime="2026-05-25 16:48:33.624985641 +0000 UTC m=+718.972903890"
May 25 16:48:39.579572 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:39.579543    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-bb4f4487f-kr6cm"
May 25 16:48:44.575745 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:44.575717    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-6qnq8"
May 25 16:48:55.777012 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:55.776976    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"]
May 25 16:48:55.782213 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:55.782192    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:55.786099 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:55.786070    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
May 25 16:48:55.786230 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:55.786107    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
May 25 16:48:55.786230 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:55.786070    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5kg6j\""
May 25 16:48:55.786230 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:55.786086    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
May 25 16:48:55.790504 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:55.790480    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"]
May 25 16:48:55.906251 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:55.906208    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:55.906425 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:55.906278    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5csc\" (UniqueName: \"kubernetes.io/projected/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-kube-api-access-d5csc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:55.906425 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:55.906317    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:55.906425 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:55.906392    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:55.906527 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:55.906440    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:55.906527 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:55.906479    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:56.007565 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:56.007522    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:56.007745 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:56.007578    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:56.007745 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:56.007607    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:56.007745 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:56.007652    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d5csc\" (UniqueName: \"kubernetes.io/projected/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-kube-api-access-d5csc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:56.007745 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:56.007693    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:56.007745 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:56.007743    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:56.008117 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:56.008091    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:56.008247 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:56.008126    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:56.008247 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:56.008169    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:56.009938 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:56.009920    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:56.010246 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:56.010231    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:56.018281 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:56.018252    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5csc\" (UniqueName: \"kubernetes.io/projected/508c630b-8cb9-4a5b-ab76-ad681a4e8ff4-kube-api-access-d5csc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f\" (UID: \"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:56.093589 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:56.093492    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:48:56.235919 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:56.235893    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"]
May 25 16:48:56.238641 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:48:56.238613    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod508c630b_8cb9_4a5b_ab76_ad681a4e8ff4.slice/crio-368fd4ffbb7ffae598041c0de251c85da6487739c08500f912e6e45821c268e4 WatchSource:0}: Error finding container 368fd4ffbb7ffae598041c0de251c85da6487739c08500f912e6e45821c268e4: Status 404 returned error can't find the container with id 368fd4ffbb7ffae598041c0de251c85da6487739c08500f912e6e45821c268e4
May 25 16:48:56.651560 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:48:56.651521    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f" event={"ID":"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4","Type":"ContainerStarted","Data":"368fd4ffbb7ffae598041c0de251c85da6487739c08500f912e6e45821c268e4"}
May 25 16:49:00.364688 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.364648    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"]
May 25 16:49:00.387844 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.387663    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"]
May 25 16:49:00.387844 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.387808    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.390914 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.390891    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
May 25 16:49:00.553439 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.553378    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9436c9da-e76c-4230-ae9d-ca3aa7a35775-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.553439 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.553431    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9436c9da-e76c-4230-ae9d-ca3aa7a35775-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.554663 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.553466    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9436c9da-e76c-4230-ae9d-ca3aa7a35775-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.554663 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.553550    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9436c9da-e76c-4230-ae9d-ca3aa7a35775-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.554663 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.553611    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ggvw\" (UniqueName: \"kubernetes.io/projected/9436c9da-e76c-4230-ae9d-ca3aa7a35775-kube-api-access-5ggvw\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.554663 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.553687    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9436c9da-e76c-4230-ae9d-ca3aa7a35775-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.655057 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.654971    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9436c9da-e76c-4230-ae9d-ca3aa7a35775-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.655057 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.655020    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9436c9da-e76c-4230-ae9d-ca3aa7a35775-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.655057 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.655040    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9436c9da-e76c-4230-ae9d-ca3aa7a35775-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.655331 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.655070    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9436c9da-e76c-4230-ae9d-ca3aa7a35775-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.655331 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.655100    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9436c9da-e76c-4230-ae9d-ca3aa7a35775-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.655331 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.655146    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5ggvw\" (UniqueName: \"kubernetes.io/projected/9436c9da-e76c-4230-ae9d-ca3aa7a35775-kube-api-access-5ggvw\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.655478 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.655402    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9436c9da-e76c-4230-ae9d-ca3aa7a35775-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.655631 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.655609    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9436c9da-e76c-4230-ae9d-ca3aa7a35775-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.655781 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.655741    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9436c9da-e76c-4230-ae9d-ca3aa7a35775-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.657718 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.657644    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9436c9da-e76c-4230-ae9d-ca3aa7a35775-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.658225 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.658204    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9436c9da-e76c-4230-ae9d-ca3aa7a35775-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.665308 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.665263    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ggvw\" (UniqueName: \"kubernetes.io/projected/9436c9da-e76c-4230-ae9d-ca3aa7a35775-kube-api-access-5ggvw\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g\" (UID: \"9436c9da-e76c-4230-ae9d-ca3aa7a35775\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:00.701053 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:00.701015    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:02.154692 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:02.154661    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"]
May 25 16:49:02.157312 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:49:02.157280    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9436c9da_e76c_4230_ae9d_ca3aa7a35775.slice/crio-ca6ef198e97456d885fe85aa9f1abb53754432d655457e96f07b7431f9b2581a WatchSource:0}: Error finding container ca6ef198e97456d885fe85aa9f1abb53754432d655457e96f07b7431f9b2581a: Status 404 returned error can't find the container with id ca6ef198e97456d885fe85aa9f1abb53754432d655457e96f07b7431f9b2581a
May 25 16:49:02.160964 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:02.160944    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 16:49:02.674223 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:02.674178    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g" event={"ID":"9436c9da-e76c-4230-ae9d-ca3aa7a35775","Type":"ContainerStarted","Data":"858d54b26bd91849c40303c0007ee8b3d9d4b378e93e29fadda64796e57004f0"}
May 25 16:49:02.674435 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:02.674230    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g" event={"ID":"9436c9da-e76c-4230-ae9d-ca3aa7a35775","Type":"ContainerStarted","Data":"ca6ef198e97456d885fe85aa9f1abb53754432d655457e96f07b7431f9b2581a"}
May 25 16:49:02.675750 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:02.675718    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f" event={"ID":"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4","Type":"ContainerStarted","Data":"54f7c85f4046cffbd9c0dc783dc421eb10f0dfde615675bc8eaef3372ec0bcb9"}
May 25 16:49:08.700679 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:08.700642    2554 generic.go:358] "Generic (PLEG): container finished" podID="508c630b-8cb9-4a5b-ab76-ad681a4e8ff4" containerID="54f7c85f4046cffbd9c0dc783dc421eb10f0dfde615675bc8eaef3372ec0bcb9" exitCode=0
May 25 16:49:08.701138 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:08.700720    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f" event={"ID":"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4","Type":"ContainerDied","Data":"54f7c85f4046cffbd9c0dc783dc421eb10f0dfde615675bc8eaef3372ec0bcb9"}
May 25 16:49:08.702159 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:08.702133    2554 generic.go:358] "Generic (PLEG): container finished" podID="9436c9da-e76c-4230-ae9d-ca3aa7a35775" containerID="858d54b26bd91849c40303c0007ee8b3d9d4b378e93e29fadda64796e57004f0" exitCode=0
May 25 16:49:08.702279 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:08.702196    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g" event={"ID":"9436c9da-e76c-4230-ae9d-ca3aa7a35775","Type":"ContainerDied","Data":"858d54b26bd91849c40303c0007ee8b3d9d4b378e93e29fadda64796e57004f0"}
May 25 16:49:10.710854 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:10.710820    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g" event={"ID":"9436c9da-e76c-4230-ae9d-ca3aa7a35775","Type":"ContainerStarted","Data":"71f326073564641e0fd1db2a658b1731a8c10bdaa142a9b00fd7dda786efba15"}
May 25 16:49:10.711337 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:10.711050    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:10.712398 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:10.712375    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f" event={"ID":"508c630b-8cb9-4a5b-ab76-ad681a4e8ff4","Type":"ContainerStarted","Data":"2316ed8a5b0b7a030753236626cdea1b202cb5366d9e8dbc2ed71bb69f92f146"}
May 25 16:49:10.712566 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:10.712552    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:49:10.732715 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:10.732663    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g" podStartSLOduration=9.461931863 podStartE2EDuration="10.732647781s" podCreationTimestamp="2026-05-25 16:49:00 +0000 UTC" firstStartedPulling="2026-05-25 16:49:08.702748177 +0000 UTC m=+754.050666405" lastFinishedPulling="2026-05-25 16:49:09.973464086 +0000 UTC m=+755.321382323" observedRunningTime="2026-05-25 16:49:10.730044778 +0000 UTC m=+756.077963030" watchObservedRunningTime="2026-05-25 16:49:10.732647781 +0000 UTC m=+756.080566030"
May 25 16:49:10.748517 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:10.748466    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f" podStartSLOduration=2.009571544 podStartE2EDuration="15.748451985s" podCreationTimestamp="2026-05-25 16:48:55 +0000 UTC" firstStartedPulling="2026-05-25 16:48:56.240412946 +0000 UTC m=+741.588331175" lastFinishedPulling="2026-05-25 16:49:09.979293382 +0000 UTC m=+755.327211616" observedRunningTime="2026-05-25 16:49:10.748017857 +0000 UTC m=+756.095936108" watchObservedRunningTime="2026-05-25 16:49:10.748451985 +0000 UTC m=+756.096370235"
May 25 16:49:17.366318 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.366272    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"]
May 25 16:49:17.372407 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.372381    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.375090 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.375067    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
May 25 16:49:17.380309 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.380286    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"]
May 25 16:49:17.500024 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.499988    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqnp6\" (UniqueName: \"kubernetes.io/projected/3da66db5-f381-4cea-b7cd-ab27e97201f0-kube-api-access-fqnp6\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.500220 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.500038    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3da66db5-f381-4cea-b7cd-ab27e97201f0-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.500220 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.500121    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3da66db5-f381-4cea-b7cd-ab27e97201f0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.500220 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.500171    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3da66db5-f381-4cea-b7cd-ab27e97201f0-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.500220 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.500189    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3da66db5-f381-4cea-b7cd-ab27e97201f0-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.500403 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.500266    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3da66db5-f381-4cea-b7cd-ab27e97201f0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.601275 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.601234    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3da66db5-f381-4cea-b7cd-ab27e97201f0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.601497 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.601292    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3da66db5-f381-4cea-b7cd-ab27e97201f0-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.601497 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.601317    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3da66db5-f381-4cea-b7cd-ab27e97201f0-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.601497 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.601353    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3da66db5-f381-4cea-b7cd-ab27e97201f0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.601667 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.601533    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fqnp6\" (UniqueName: \"kubernetes.io/projected/3da66db5-f381-4cea-b7cd-ab27e97201f0-kube-api-access-fqnp6\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.601667 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.601618    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3da66db5-f381-4cea-b7cd-ab27e97201f0-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.601776 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.601712    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3da66db5-f381-4cea-b7cd-ab27e97201f0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.601776 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.601735    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3da66db5-f381-4cea-b7cd-ab27e97201f0-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.601919 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.601776    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3da66db5-f381-4cea-b7cd-ab27e97201f0-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.603932 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.603904    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3da66db5-f381-4cea-b7cd-ab27e97201f0-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.604163 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.604148    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3da66db5-f381-4cea-b7cd-ab27e97201f0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.609928 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.609902    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqnp6\" (UniqueName: \"kubernetes.io/projected/3da66db5-f381-4cea-b7cd-ab27e97201f0-kube-api-access-fqnp6\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2\" (UID: \"3da66db5-f381-4cea-b7cd-ab27e97201f0\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.683402 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.683304    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:17.858062 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:17.858037    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"]
May 25 16:49:17.860295 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:49:17.860267    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3da66db5_f381_4cea_b7cd_ab27e97201f0.slice/crio-e07c52d30581d21b34ca341947d79e9f41074f2bdb0aa484319bcdeb49a81362 WatchSource:0}: Error finding container e07c52d30581d21b34ca341947d79e9f41074f2bdb0aa484319bcdeb49a81362: Status 404 returned error can't find the container with id e07c52d30581d21b34ca341947d79e9f41074f2bdb0aa484319bcdeb49a81362
May 25 16:49:18.743631 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:18.743590    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2" event={"ID":"3da66db5-f381-4cea-b7cd-ab27e97201f0","Type":"ContainerStarted","Data":"bbf5a5366cecfc0e2639ac56d6d4eb57c4fc49d6b554378cd7002eee3b8005c8"}
May 25 16:49:18.744039 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:18.743637    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2" event={"ID":"3da66db5-f381-4cea-b7cd-ab27e97201f0","Type":"ContainerStarted","Data":"e07c52d30581d21b34ca341947d79e9f41074f2bdb0aa484319bcdeb49a81362"}
May 25 16:49:21.732677 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:21.732646    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g"
May 25 16:49:21.734778 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:21.734758    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f"
May 25 16:49:23.260515 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.260474    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"]
May 25 16:49:23.266370 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.266334    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.269176 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.269154    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
May 25 16:49:23.275998 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.275965    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"]
May 25 16:49:23.354060 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.354018    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtnt2\" (UniqueName: \"kubernetes.io/projected/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-kube-api-access-qtnt2\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.354249 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.354082    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.354249 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.354141    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.354249 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.354212    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.354411 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.354265    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.354411 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.354312    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.454916 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.454878    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.455105 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.454931    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.455105 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.454957    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.455105 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.454986    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.455105 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.455012    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.455105 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.455062    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qtnt2\" (UniqueName: \"kubernetes.io/projected/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-kube-api-access-qtnt2\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.455364 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.455302    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.455460 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.455435    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.455505 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.455469    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.457858 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.457831    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.458149 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.458121    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.463727 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.463693    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtnt2\" (UniqueName: \"kubernetes.io/projected/54dd2ea7-e4a7-43ba-815c-2c130a4efd30-kube-api-access-qtnt2\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-lszqm\" (UID: \"54dd2ea7-e4a7-43ba-815c-2c130a4efd30\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.578853 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.578806    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:23.722361 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.720877    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"]
May 25 16:49:23.728214 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:49:23.728182    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54dd2ea7_e4a7_43ba_815c_2c130a4efd30.slice/crio-078f708423f1c4a7b27bb157fe3e67576c99a03bd71a75f89d8fe148b63e16c4 WatchSource:0}: Error finding container 078f708423f1c4a7b27bb157fe3e67576c99a03bd71a75f89d8fe148b63e16c4: Status 404 returned error can't find the container with id 078f708423f1c4a7b27bb157fe3e67576c99a03bd71a75f89d8fe148b63e16c4
May 25 16:49:23.763517 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.763484    2554 generic.go:358] "Generic (PLEG): container finished" podID="3da66db5-f381-4cea-b7cd-ab27e97201f0" containerID="bbf5a5366cecfc0e2639ac56d6d4eb57c4fc49d6b554378cd7002eee3b8005c8" exitCode=0
May 25 16:49:23.763692 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.763563    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2" event={"ID":"3da66db5-f381-4cea-b7cd-ab27e97201f0","Type":"ContainerDied","Data":"bbf5a5366cecfc0e2639ac56d6d4eb57c4fc49d6b554378cd7002eee3b8005c8"}
May 25 16:49:23.764768 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:23.764742    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm" event={"ID":"54dd2ea7-e4a7-43ba-815c-2c130a4efd30","Type":"ContainerStarted","Data":"078f708423f1c4a7b27bb157fe3e67576c99a03bd71a75f89d8fe148b63e16c4"}
May 25 16:49:24.772133 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:24.772098    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm" event={"ID":"54dd2ea7-e4a7-43ba-815c-2c130a4efd30","Type":"ContainerStarted","Data":"70e86566dc4453dc10346eea17b9e4655357a22253f5a0ce5338a392a4e625f8"}
May 25 16:49:24.773994 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:24.773965    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2" event={"ID":"3da66db5-f381-4cea-b7cd-ab27e97201f0","Type":"ContainerStarted","Data":"09ddc40e5de275bb2b9ee78dee6db209d7d4d18d8a93b6ff959df4d60ab3f677"}
May 25 16:49:24.774210 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:24.774181    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:24.807463 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:24.807411    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2" podStartSLOduration=7.593266274 podStartE2EDuration="7.807394901s" podCreationTimestamp="2026-05-25 16:49:17 +0000 UTC" firstStartedPulling="2026-05-25 16:49:23.764279449 +0000 UTC m=+769.112197677" lastFinishedPulling="2026-05-25 16:49:23.978408058 +0000 UTC m=+769.326326304" observedRunningTime="2026-05-25 16:49:24.805799666 +0000 UTC m=+770.153717917" watchObservedRunningTime="2026-05-25 16:49:24.807394901 +0000 UTC m=+770.155313150"
May 25 16:49:32.807533 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:32.807500    2554 generic.go:358] "Generic (PLEG): container finished" podID="54dd2ea7-e4a7-43ba-815c-2c130a4efd30" containerID="70e86566dc4453dc10346eea17b9e4655357a22253f5a0ce5338a392a4e625f8" exitCode=0
May 25 16:49:32.807939 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:32.807574    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm" event={"ID":"54dd2ea7-e4a7-43ba-815c-2c130a4efd30","Type":"ContainerDied","Data":"70e86566dc4453dc10346eea17b9e4655357a22253f5a0ce5338a392a4e625f8"}
May 25 16:49:33.813515 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:33.813476    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm" event={"ID":"54dd2ea7-e4a7-43ba-815c-2c130a4efd30","Type":"ContainerStarted","Data":"0edf9479494fc05f829c3f8bb250218776099a2dc2e5419517a5439d3b0a3700"}
May 25 16:49:33.813957 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:33.813669    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:49:33.833260 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:33.833193    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm" podStartSLOduration=10.657108575 podStartE2EDuration="10.833177466s" podCreationTimestamp="2026-05-25 16:49:23 +0000 UTC" firstStartedPulling="2026-05-25 16:49:32.808200234 +0000 UTC m=+778.156118463" lastFinishedPulling="2026-05-25 16:49:32.984269124 +0000 UTC m=+778.332187354" observedRunningTime="2026-05-25 16:49:33.830662048 +0000 UTC m=+779.178580297" watchObservedRunningTime="2026-05-25 16:49:33.833177466 +0000 UTC m=+779.181095716"
May 25 16:49:35.791156 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:35.791121    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2"
May 25 16:49:44.832725 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:49:44.832686    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-lszqm"
May 25 16:51:31.321144 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:31.321051    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b757cc664-sqc4q"]
May 25 16:51:31.321722 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:31.321408    2554 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-7b757cc664-sqc4q" podUID="81859385-a343-4abb-96e8-6450a18b17e3" containerName="manager" containerID="cri-o://33e5cb39f3f3a2ccbe97648df47ba13ddd34970efe19e5056cf921dfcfe2967c" gracePeriod=30
May 25 16:51:31.570088 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:31.570061    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b757cc664-sqc4q"
May 25 16:51:31.669462 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:31.669370    2554 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dhvj\" (UniqueName: \"kubernetes.io/projected/81859385-a343-4abb-96e8-6450a18b17e3-kube-api-access-8dhvj\") pod \"81859385-a343-4abb-96e8-6450a18b17e3\" (UID: \"81859385-a343-4abb-96e8-6450a18b17e3\") "
May 25 16:51:31.671514 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:31.671478    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81859385-a343-4abb-96e8-6450a18b17e3-kube-api-access-8dhvj" (OuterVolumeSpecName: "kube-api-access-8dhvj") pod "81859385-a343-4abb-96e8-6450a18b17e3" (UID: "81859385-a343-4abb-96e8-6450a18b17e3"). InnerVolumeSpecName "kube-api-access-8dhvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 16:51:31.770245 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:31.770205    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dhvj\" (UniqueName: \"kubernetes.io/projected/81859385-a343-4abb-96e8-6450a18b17e3-kube-api-access-8dhvj\") on node \"ip-10-0-135-137.ec2.internal\" DevicePath \"\""
May 25 16:51:32.237014 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:32.236981    2554 generic.go:358] "Generic (PLEG): container finished" podID="81859385-a343-4abb-96e8-6450a18b17e3" containerID="33e5cb39f3f3a2ccbe97648df47ba13ddd34970efe19e5056cf921dfcfe2967c" exitCode=0
May 25 16:51:32.237198 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:32.237043    2554 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b757cc664-sqc4q"
May 25 16:51:32.237198 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:32.237073    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b757cc664-sqc4q" event={"ID":"81859385-a343-4abb-96e8-6450a18b17e3","Type":"ContainerDied","Data":"33e5cb39f3f3a2ccbe97648df47ba13ddd34970efe19e5056cf921dfcfe2967c"}
May 25 16:51:32.237198 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:32.237113    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b757cc664-sqc4q" event={"ID":"81859385-a343-4abb-96e8-6450a18b17e3","Type":"ContainerDied","Data":"57f97ad16b446b5ea4630ef6c90210804880f7f1b8f0ec675ab24f898f143b79"}
May 25 16:51:32.237198 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:32.237131    2554 scope.go:117] "RemoveContainer" containerID="33e5cb39f3f3a2ccbe97648df47ba13ddd34970efe19e5056cf921dfcfe2967c"
May 25 16:51:32.246081 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:32.246061    2554 scope.go:117] "RemoveContainer" containerID="33e5cb39f3f3a2ccbe97648df47ba13ddd34970efe19e5056cf921dfcfe2967c"
May 25 16:51:32.246362 ip-10-0-135-137 kubenswrapper[2554]: E0525 16:51:32.246343    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33e5cb39f3f3a2ccbe97648df47ba13ddd34970efe19e5056cf921dfcfe2967c\": container with ID starting with 33e5cb39f3f3a2ccbe97648df47ba13ddd34970efe19e5056cf921dfcfe2967c not found: ID does not exist" containerID="33e5cb39f3f3a2ccbe97648df47ba13ddd34970efe19e5056cf921dfcfe2967c"
May 25 16:51:32.246423 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:32.246370    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33e5cb39f3f3a2ccbe97648df47ba13ddd34970efe19e5056cf921dfcfe2967c"} err="failed to get container status \"33e5cb39f3f3a2ccbe97648df47ba13ddd34970efe19e5056cf921dfcfe2967c\": rpc error: code = NotFound desc = could not find container \"33e5cb39f3f3a2ccbe97648df47ba13ddd34970efe19e5056cf921dfcfe2967c\": container with ID starting with 33e5cb39f3f3a2ccbe97648df47ba13ddd34970efe19e5056cf921dfcfe2967c not found: ID does not exist"
May 25 16:51:32.260048 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:32.260016    2554 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b757cc664-sqc4q"]
May 25 16:51:32.266896 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:32.266764    2554 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7b757cc664-sqc4q"]
May 25 16:51:33.020221 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:33.020181    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b757cc664-sb8st"]
May 25 16:51:33.020612 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:33.020547    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81859385-a343-4abb-96e8-6450a18b17e3" containerName="manager"
May 25 16:51:33.020612 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:33.020557    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="81859385-a343-4abb-96e8-6450a18b17e3" containerName="manager"
May 25 16:51:33.020690 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:33.020618    2554 memory_manager.go:356] "RemoveStaleState removing state" podUID="81859385-a343-4abb-96e8-6450a18b17e3" containerName="manager"
May 25 16:51:33.024815 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:33.024797    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b757cc664-sb8st"
May 25 16:51:33.027729 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:33.027704    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-sdnrv\""
May 25 16:51:33.027853 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:33.027746    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
May 25 16:51:33.032537 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:33.032512    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b757cc664-sb8st"]
May 25 16:51:33.183636 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:33.183600    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w429w\" (UniqueName: \"kubernetes.io/projected/94b939a0-ce82-4d6e-b7b4-a50562a6dc27-kube-api-access-w429w\") pod \"maas-controller-7b757cc664-sb8st\" (UID: \"94b939a0-ce82-4d6e-b7b4-a50562a6dc27\") " pod="opendatahub/maas-controller-7b757cc664-sb8st"
May 25 16:51:33.267403 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:33.267372    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81859385-a343-4abb-96e8-6450a18b17e3" path="/var/lib/kubelet/pods/81859385-a343-4abb-96e8-6450a18b17e3/volumes"
May 25 16:51:33.285070 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:33.285000    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w429w\" (UniqueName: \"kubernetes.io/projected/94b939a0-ce82-4d6e-b7b4-a50562a6dc27-kube-api-access-w429w\") pod \"maas-controller-7b757cc664-sb8st\" (UID: \"94b939a0-ce82-4d6e-b7b4-a50562a6dc27\") " pod="opendatahub/maas-controller-7b757cc664-sb8st"
May 25 16:51:33.293825 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:33.293785    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w429w\" (UniqueName: \"kubernetes.io/projected/94b939a0-ce82-4d6e-b7b4-a50562a6dc27-kube-api-access-w429w\") pod \"maas-controller-7b757cc664-sb8st\" (UID: \"94b939a0-ce82-4d6e-b7b4-a50562a6dc27\") " pod="opendatahub/maas-controller-7b757cc664-sb8st"
May 25 16:51:33.337343 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:33.337285    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b757cc664-sb8st"
May 25 16:51:33.475758 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:33.475685    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b757cc664-sb8st"]
May 25 16:51:33.478584 ip-10-0-135-137 kubenswrapper[2554]: W0525 16:51:33.478544    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94b939a0_ce82_4d6e_b7b4_a50562a6dc27.slice/crio-14c9aa24bb92b56fe2d14b69e00b33b506fc64f3866b92741c65547ef32df010 WatchSource:0}: Error finding container 14c9aa24bb92b56fe2d14b69e00b33b506fc64f3866b92741c65547ef32df010: Status 404 returned error can't find the container with id 14c9aa24bb92b56fe2d14b69e00b33b506fc64f3866b92741c65547ef32df010
May 25 16:51:34.246269 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:34.246180    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b757cc664-sb8st" event={"ID":"94b939a0-ce82-4d6e-b7b4-a50562a6dc27","Type":"ContainerStarted","Data":"00ca537f300cdb616a784f597e609f402b29f2454c98998400c0eecede71d0de"}
May 25 16:51:34.246269 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:34.246218    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b757cc664-sb8st" event={"ID":"94b939a0-ce82-4d6e-b7b4-a50562a6dc27","Type":"ContainerStarted","Data":"14c9aa24bb92b56fe2d14b69e00b33b506fc64f3866b92741c65547ef32df010"}
May 25 16:51:34.246687 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:34.246297    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b757cc664-sb8st"
May 25 16:51:34.265271 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:34.265225    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-7b757cc664-sb8st" podStartSLOduration=0.8519379 podStartE2EDuration="1.265209901s" podCreationTimestamp="2026-05-25 16:51:33 +0000 UTC" firstStartedPulling="2026-05-25 16:51:33.479902667 +0000 UTC m=+898.827820895" lastFinishedPulling="2026-05-25 16:51:33.893174665 +0000 UTC m=+899.241092896" observedRunningTime="2026-05-25 16:51:34.262816691 +0000 UTC m=+899.610734952" watchObservedRunningTime="2026-05-25 16:51:34.265209901 +0000 UTC m=+899.613128149"
May 25 16:51:35.215056 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:35.215028    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 16:51:35.216293 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:35.216268    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 16:51:45.255587 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:51:45.255548    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b757cc664-sb8st"
May 25 16:56:35.241021 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:56:35.240992    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 16:56:35.242917 ip-10-0-135-137 kubenswrapper[2554]: I0525 16:56:35.242896    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 17:01:35.271957 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:01:35.271928    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 17:01:35.275458 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:01:35.275437    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 17:06:35.294358 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:06:35.294328    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 17:06:35.299033 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:06:35.299010    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 17:11:35.317679 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:11:35.317641    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 17:11:35.323347 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:11:35.323324    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ggvbg_230e7d42-ab66-49b4-876c-969beac2ebca/ovn-acl-logging/0.log"
May 25 17:12:54.281710 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:54.281631    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-9b78t_de84230f-ae26-4d43-9851-e9682f74daa8/manager/0.log"
May 25 17:12:54.403773 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:54.403742    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-bb4f4487f-kr6cm_cd2325eb-4604-4461-bbd4-29b885a3272a/maas-api/0.log"
May 25 17:12:54.647166 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:54.647094    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7b757cc664-sb8st_94b939a0-ce82-4d6e-b7b4-a50562a6dc27/manager/0.log"
May 25 17:12:54.767739 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:54.767708    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-6lt69_b6e4679e-daaf-4547-971d-b0522e3dd7ec/manager/2.log"
May 25 17:12:54.885006 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:54.884975    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86d64d44d9-tpm2q_906d0423-c29f-4f77-bb1b-f284cf6b6c8f/manager/0.log"
May 25 17:12:55.239677 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:55.239643    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-wmpqx_cb4ccb90-d2c9-437d-b38c-b6496ab33948/postgres/0.log"
May 25 17:12:56.788285 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:56.788256    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-xk7bb_66ae27e7-5f35-40eb-9596-796730721c49/manager/0.log"
May 25 17:12:57.374298 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:57.374268    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-w89xq_b5fbd335-caf1-4bee-bcd3-ff3abedc59cd/manager/0.log"
May 25 17:12:57.836002 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:57.835954    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-qhfrf_3c165474-1178-4dc1-a2b8-29b7958369b0/discovery/0.log"
May 25 17:12:58.076986 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:58.076957    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77d58df487-cwxfg_b886e360-1d17-4718-9ae4-db58180afb80/kube-auth-proxy/0.log"
May 25 17:12:58.381071 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:58.381040    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-6qnq8_eaa49daf-ccbd-42a0-9b73-1c4bc8ea883f/payload-processing/0.log"
May 25 17:12:58.850212 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:58.850180    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2_3da66db5-f381-4cea-b7cd-ab27e97201f0/storage-initializer/0.log"
May 25 17:12:58.859284 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:58.859262    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-rm5m2_3da66db5-f381-4cea-b7cd-ab27e97201f0/main/0.log"
May 25 17:12:59.226768 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:59.226683    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f_508c630b-8cb9-4a5b-ab76-ad681a4e8ff4/storage-initializer/0.log"
May 25 17:12:59.234773 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:59.234748    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccg4f5f_508c630b-8cb9-4a5b-ab76-ad681a4e8ff4/main/0.log"
May 25 17:12:59.345113 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:59.345087    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g_9436c9da-e76c-4230-ae9d-ca3aa7a35775/main/0.log"
May 25 17:12:59.352193 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:59.352168    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-fzx9g_9436c9da-e76c-4230-ae9d-ca3aa7a35775/storage-initializer/0.log"
May 25 17:12:59.464993 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:59.464962    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-lszqm_54dd2ea7-e4a7-43ba-815c-2c130a4efd30/storage-initializer/0.log"
May 25 17:12:59.472814 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:12:59.472791    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-lszqm_54dd2ea7-e4a7-43ba-815c-2c130a4efd30/main/0.log"
May 25 17:13:06.321993 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:06.321965    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bdqw9_eb80c465-3f53-4d73-abef-d8465200fa2c/global-pull-secret-syncer/0.log"
May 25 17:13:06.488879 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:06.488827    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xfhkv_1018b970-9e3e-4eb4-a045-17340c9ea537/konnectivity-agent/0.log"
May 25 17:13:06.530385 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:06.530355    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-137.ec2.internal_9e7d79ffc7bf90f02369e701a94a0e0a/haproxy/0.log"
May 25 17:13:11.382981 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:11.382951    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-xk7bb_66ae27e7-5f35-40eb-9596-796730721c49/manager/0.log"
May 25 17:13:11.648403 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:11.648292    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-w89xq_b5fbd335-caf1-4bee-bcd3-ff3abedc59cd/manager/0.log"
May 25 17:13:13.668921 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:13.668891    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mwjld_51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a/node-exporter/0.log"
May 25 17:13:13.707264 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:13.707232    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mwjld_51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a/kube-rbac-proxy/0.log"
May 25 17:13:13.743882 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:13.743851    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mwjld_51e96ef1-e2ed-49dc-ab7f-d6bdec2ef69a/init-textfile/0.log"
May 25 17:13:14.503969 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.503931    2554 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"]
May 25 17:13:14.513090 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.513059    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.516779 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.516752    2554 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7rhsv\"/\"default-dockercfg-8wzww\""
May 25 17:13:14.517756 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.517737    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7rhsv\"/\"kube-root-ca.crt\""
May 25 17:13:14.517954 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.517935    2554 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7rhsv\"/\"openshift-service-ca.crt\""
May 25 17:13:14.533306 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.533278    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"]
May 25 17:13:14.605414 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.605380    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9ecb2810-ba99-4e0c-b45a-5900de518f53-proc\") pod \"perf-node-gather-daemonset-zg2bp\" (UID: \"9ecb2810-ba99-4e0c-b45a-5900de518f53\") " pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.605414 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.605420    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lkcr\" (UniqueName: \"kubernetes.io/projected/9ecb2810-ba99-4e0c-b45a-5900de518f53-kube-api-access-9lkcr\") pod \"perf-node-gather-daemonset-zg2bp\" (UID: \"9ecb2810-ba99-4e0c-b45a-5900de518f53\") " pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.605645 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.605486    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9ecb2810-ba99-4e0c-b45a-5900de518f53-podres\") pod \"perf-node-gather-daemonset-zg2bp\" (UID: \"9ecb2810-ba99-4e0c-b45a-5900de518f53\") " pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.605645 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.605550    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ecb2810-ba99-4e0c-b45a-5900de518f53-sys\") pod \"perf-node-gather-daemonset-zg2bp\" (UID: \"9ecb2810-ba99-4e0c-b45a-5900de518f53\") " pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.605645 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.605606    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ecb2810-ba99-4e0c-b45a-5900de518f53-lib-modules\") pod \"perf-node-gather-daemonset-zg2bp\" (UID: \"9ecb2810-ba99-4e0c-b45a-5900de518f53\") " pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.706945 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.706896    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ecb2810-ba99-4e0c-b45a-5900de518f53-lib-modules\") pod \"perf-node-gather-daemonset-zg2bp\" (UID: \"9ecb2810-ba99-4e0c-b45a-5900de518f53\") " pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.706945 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.706955    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9ecb2810-ba99-4e0c-b45a-5900de518f53-proc\") pod \"perf-node-gather-daemonset-zg2bp\" (UID: \"9ecb2810-ba99-4e0c-b45a-5900de518f53\") " pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.707375 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.706973    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9lkcr\" (UniqueName: \"kubernetes.io/projected/9ecb2810-ba99-4e0c-b45a-5900de518f53-kube-api-access-9lkcr\") pod \"perf-node-gather-daemonset-zg2bp\" (UID: \"9ecb2810-ba99-4e0c-b45a-5900de518f53\") " pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.707375 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.706998    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9ecb2810-ba99-4e0c-b45a-5900de518f53-podres\") pod \"perf-node-gather-daemonset-zg2bp\" (UID: \"9ecb2810-ba99-4e0c-b45a-5900de518f53\") " pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.707375 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.707036    2554 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ecb2810-ba99-4e0c-b45a-5900de518f53-sys\") pod \"perf-node-gather-daemonset-zg2bp\" (UID: \"9ecb2810-ba99-4e0c-b45a-5900de518f53\") " pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.707375 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.707062    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ecb2810-ba99-4e0c-b45a-5900de518f53-lib-modules\") pod \"perf-node-gather-daemonset-zg2bp\" (UID: \"9ecb2810-ba99-4e0c-b45a-5900de518f53\") " pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.707375 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.707089    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9ecb2810-ba99-4e0c-b45a-5900de518f53-proc\") pod \"perf-node-gather-daemonset-zg2bp\" (UID: \"9ecb2810-ba99-4e0c-b45a-5900de518f53\") " pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.707375 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.707117    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ecb2810-ba99-4e0c-b45a-5900de518f53-sys\") pod \"perf-node-gather-daemonset-zg2bp\" (UID: \"9ecb2810-ba99-4e0c-b45a-5900de518f53\") " pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.707375 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.707138    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9ecb2810-ba99-4e0c-b45a-5900de518f53-podres\") pod \"perf-node-gather-daemonset-zg2bp\" (UID: \"9ecb2810-ba99-4e0c-b45a-5900de518f53\") " pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.722180 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.722149    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lkcr\" (UniqueName: \"kubernetes.io/projected/9ecb2810-ba99-4e0c-b45a-5900de518f53-kube-api-access-9lkcr\") pod \"perf-node-gather-daemonset-zg2bp\" (UID: \"9ecb2810-ba99-4e0c-b45a-5900de518f53\") " pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.823084 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.822998    2554 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:14.980930 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.980857    2554 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"]
May 25 17:13:14.983816 ip-10-0-135-137 kubenswrapper[2554]: W0525 17:13:14.983788    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9ecb2810_ba99_4e0c_b45a_5900de518f53.slice/crio-ea2e836818d57249f3d1a7cdd248b272cb86983d927b3cc1eab3d39fd61e6f2d WatchSource:0}: Error finding container ea2e836818d57249f3d1a7cdd248b272cb86983d927b3cc1eab3d39fd61e6f2d: Status 404 returned error can't find the container with id ea2e836818d57249f3d1a7cdd248b272cb86983d927b3cc1eab3d39fd61e6f2d
May 25 17:13:14.985430 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:14.985412    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 17:13:15.744689 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:15.744656    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp" event={"ID":"9ecb2810-ba99-4e0c-b45a-5900de518f53","Type":"ContainerStarted","Data":"10d7937cff1fbfff70f5f84c60b2b9a45c15da5d4ea616eac2f46ae4bde3888c"}
May 25 17:13:15.744689 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:15.744691    2554 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp" event={"ID":"9ecb2810-ba99-4e0c-b45a-5900de518f53","Type":"ContainerStarted","Data":"ea2e836818d57249f3d1a7cdd248b272cb86983d927b3cc1eab3d39fd61e6f2d"}
May 25 17:13:15.745126 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:15.744714    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:15.775582 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:15.775532    2554 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp" podStartSLOduration=1.7755156159999999 podStartE2EDuration="1.775515616s" podCreationTimestamp="2026-05-25 17:13:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 17:13:15.774569191 +0000 UTC m=+2201.122487441" watchObservedRunningTime="2026-05-25 17:13:15.775515616 +0000 UTC m=+2201.123433865"
May 25 17:13:15.875671 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:15.875641    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-jsgsj_02a5243d-f581-40f8-a48a-b2cfd2296e89/networking-console-plugin/0.log"
May 25 17:13:18.553505 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:18.553457    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lndmn_dba85d0a-072c-4a30-b169-92843a576b3b/dns/0.log"
May 25 17:13:18.590136 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:18.590106    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lndmn_dba85d0a-072c-4a30-b169-92843a576b3b/kube-rbac-proxy/0.log"
May 25 17:13:18.667536 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:18.667505    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bx8kp_645b9dd9-5a8a-406b-a2af-5832530e4664/dns-node-resolver/0.log"
May 25 17:13:19.297173 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:19.297139    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wp4m6_d15b78cb-69ab-4017-8bae-92669f025b54/node-ca/0.log"
May 25 17:13:20.321813 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:20.321778    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-qhfrf_3c165474-1178-4dc1-a2b8-29b7958369b0/discovery/0.log"
May 25 17:13:20.387096 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:20.387070    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77d58df487-cwxfg_b886e360-1d17-4718-9ae4-db58180afb80/kube-auth-proxy/0.log"
May 25 17:13:20.531406 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:20.531371    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-6qnq8_eaa49daf-ccbd-42a0-9b73-1c4bc8ea883f/payload-processing/0.log"
May 25 17:13:21.092146 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:21.092113    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l2p4k_7a41c3db-9f6d-4e76-9b35-839fa3b35665/serve-healthcheck-canary/0.log"
May 25 17:13:21.758937 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:21.758910    2554 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7rhsv/perf-node-gather-daemonset-zg2bp"
May 25 17:13:21.798678 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:21.798652    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vmnrg_f99e26e0-ccfa-44a2-87ca-f50d6d75b059/kube-rbac-proxy/0.log"
May 25 17:13:21.820457 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:21.820430    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vmnrg_f99e26e0-ccfa-44a2-87ca-f50d6d75b059/exporter/0.log"
May 25 17:13:21.842620 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:21.842595    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vmnrg_f99e26e0-ccfa-44a2-87ca-f50d6d75b059/extractor/0.log"
May 25 17:13:24.026590 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:24.026544    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-9b78t_de84230f-ae26-4d43-9851-e9682f74daa8/manager/0.log"
May 25 17:13:24.059106 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:24.059079    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-bb4f4487f-kr6cm_cd2325eb-4604-4461-bbd4-29b885a3272a/maas-api/0.log"
May 25 17:13:24.150668 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:24.150630    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7b757cc664-sb8st_94b939a0-ce82-4d6e-b7b4-a50562a6dc27/manager/0.log"
May 25 17:13:24.174335 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:24.174299    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-6lt69_b6e4679e-daaf-4547-971d-b0522e3dd7ec/manager/1.log"
May 25 17:13:24.184450 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:24.184417    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-6lt69_b6e4679e-daaf-4547-971d-b0522e3dd7ec/manager/2.log"
May 25 17:13:24.230880 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:24.230833    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86d64d44d9-tpm2q_906d0423-c29f-4f77-bb1b-f284cf6b6c8f/manager/0.log"
May 25 17:13:24.332159 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:24.332128    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-wmpqx_cb4ccb90-d2c9-437d-b38c-b6496ab33948/postgres/0.log"
May 25 17:13:31.804945 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:31.804913    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n7gn7_6cc91204-a4f1-4108-b4d6-dab40dbda044/kube-multus-additional-cni-plugins/0.log"
May 25 17:13:31.828459 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:31.828425    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n7gn7_6cc91204-a4f1-4108-b4d6-dab40dbda044/egress-router-binary-copy/0.log"
May 25 17:13:31.850581 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:31.850553    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n7gn7_6cc91204-a4f1-4108-b4d6-dab40dbda044/cni-plugins/0.log"
May 25 17:13:31.872714 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:31.872690    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n7gn7_6cc91204-a4f1-4108-b4d6-dab40dbda044/bond-cni-plugin/0.log"
May 25 17:13:31.895213 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:31.895187    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n7gn7_6cc91204-a4f1-4108-b4d6-dab40dbda044/routeoverride-cni/0.log"
May 25 17:13:31.917860 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:31.917831    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n7gn7_6cc91204-a4f1-4108-b4d6-dab40dbda044/whereabouts-cni-bincopy/0.log"
May 25 17:13:31.942040 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:31.942013    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n7gn7_6cc91204-a4f1-4108-b4d6-dab40dbda044/whereabouts-cni/0.log"
May 25 17:13:31.976948 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:31.976918    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nbfsz_9e86de1e-2942-4e72-b2e7-5243b02ed554/kube-multus/0.log"
May 25 17:13:32.044699 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:32.044673    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l57q2_6646a861-65d0-4b6f-af0c-415ff46851dd/network-metrics-daemon/0.log"
May 25 17:13:32.068365 ip-10-0-135-137 kubenswrapper[2554]: I0525 17:13:32.068281    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l57q2_6646a861-65d0-4b6f-af0c-415ff46851dd/kube-rbac-proxy/0.log"