Jun 03 15:54:33.352439 ip-10-0-128-61 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 03 15:54:33.352451 ip-10-0-128-61 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 03 15:54:33.352458 ip-10-0-128-61 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 03 15:54:33.352667 ip-10-0-128-61 systemd[1]: Failed to start Kubernetes Kubelet. Jun 03 15:54:43.496122 ip-10-0-128-61 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 03 15:54:43.496143 ip-10-0-128-61 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 775f19d110014a35ac88fed9b91e9ca1 -- Jun 03 15:57:06.694125 ip-10-0-128-61 systemd[1]: Starting Kubernetes Kubelet... Jun 03 15:57:07.139420 ip-10-0-128-61 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 15:57:07.139420 ip-10-0-128-61 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 03 15:57:07.139420 ip-10-0-128-61 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 15:57:07.139420 ip-10-0-128-61 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 03 15:57:07.139420 ip-10-0-128-61 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 15:57:07.140177 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.140088 2576 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 03 15:57:07.143230 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143212 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 15:57:07.143230 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143231 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143237 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143241 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143244 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143246 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143249 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143252 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143255 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143258 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143260 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143263 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143266 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143274 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143277 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143280 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143282 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143285 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143287 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143291 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143294 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 15:57:07.143302 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143296 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143299 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143302 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143305 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143308 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143310 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143313 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143316 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143319 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143321 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143324 2576 feature_gate.go:328] unrecognized feature gate: Example Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143326 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143329 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143331 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143334 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143337 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143339 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143342 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143344 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143347 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 15:57:07.143792 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143349 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143351 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143354 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143357 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143359 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143362 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143364 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143366 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143369 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143371 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143374 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143377 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143379 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143382 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143385 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143388 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143390 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143393 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143396 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143398 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 15:57:07.144336 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143402 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143406 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143410 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143413 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143417 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143427 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143431 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143434 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143437 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143440 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143442 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143445 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143454 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143456 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143459 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143462 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143464 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143469 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143472 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 15:57:07.144854 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143474 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143477 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143480 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143483 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143485 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143488 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143888 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143894 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143897 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143900 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143903 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143905 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143908 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143910 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143913 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143915 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143918 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143921 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143924 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143927 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 15:57:07.145317 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143929 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143932 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143934 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143937 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143940 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143942 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143944 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143947 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143956 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143958 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143961 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143963 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143966 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143968 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143971 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143974 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143976 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143979 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143983 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143985 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 15:57:07.145850 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143988 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143991 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143994 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143997 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.143999 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144002 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144004 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144007 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144009 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144012 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144015 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144017 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144020 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144022 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144026 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144028 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144031 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144033 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144036 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144038 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 15:57:07.146356 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144041 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144043 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144045 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144048 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144050 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144053 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144055 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144057 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144060 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144063 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144067 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144070 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144073 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144077 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144081 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144084 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144087 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144090 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144093 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 15:57:07.146911 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144095 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144098 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144101 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144104 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144106 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144109 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144112 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144115 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144118 2576 feature_gate.go:328] unrecognized feature gate: Example Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144120 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144124 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144127 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144130 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144210 2576 flags.go:64] FLAG: --address="0.0.0.0" Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144218 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144224 2576 flags.go:64] FLAG: --anonymous-auth="true" Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144229 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144233 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144237 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144241 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144249 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 03 15:57:07.147394 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144252 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144255 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144259 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144263 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144266 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144269 2576 flags.go:64] FLAG: --cgroup-root="" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144272 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144275 2576 flags.go:64] FLAG: --client-ca-file="" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144278 2576 flags.go:64] FLAG: --cloud-config="" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144282 2576 flags.go:64] FLAG: --cloud-provider="external" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144285 2576 flags.go:64] FLAG: --cluster-dns="[]" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144290 2576 flags.go:64] FLAG: --cluster-domain="" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144292 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144296 2576 flags.go:64] FLAG: --config-dir="" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144298 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144302 2576 flags.go:64] FLAG: --container-log-max-files="5" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144306 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144309 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144312 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144316 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144319 2576 flags.go:64] FLAG: --contention-profiling="false" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144322 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144325 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144328 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144331 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 03 15:57:07.147934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144335 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144338 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144342 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144345 2576 flags.go:64] FLAG: --enable-load-reader="false" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144348 2576 flags.go:64] FLAG: --enable-server="true" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144351 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144355 2576 flags.go:64] FLAG: --event-burst="100" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144358 2576 flags.go:64] FLAG: --event-qps="50" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144361 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144365 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144368 2576 flags.go:64] FLAG: --eviction-hard="" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144371 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144374 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144377 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144380 2576 flags.go:64] FLAG: --eviction-soft="" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144383 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144386 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144389 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144392 2576 flags.go:64] FLAG: --experimental-mounter-path="" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144395 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144398 2576 flags.go:64] FLAG: --fail-swap-on="true" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144401 2576 flags.go:64] FLAG: --feature-gates="" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144405 2576 flags.go:64] FLAG: --file-check-frequency="20s" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144408 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144411 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 03 15:57:07.148557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144414 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144417 2576 flags.go:64] FLAG: --healthz-port="10248" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144420 2576 flags.go:64] FLAG: --help="false" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144423 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144427 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144430 2576 flags.go:64] FLAG: --http-check-frequency="20s" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144432 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144436 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144439 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144442 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144445 2576 flags.go:64] FLAG: --image-service-endpoint="" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144448 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144450 2576 flags.go:64] FLAG: --kube-api-burst="100" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144453 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144456 2576 flags.go:64] FLAG: --kube-api-qps="50" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144459 2576 flags.go:64] FLAG: --kube-reserved="" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144463 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144465 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144469 2576 flags.go:64] FLAG: --kubelet-cgroups="" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144472 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144475 2576 flags.go:64] FLAG: --lock-file="" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144478 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144481 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144484 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 03 15:57:07.149185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144491 2576 flags.go:64] FLAG: --log-json-split-stream="false" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144494 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144497 2576 flags.go:64] FLAG: --log-text-split-stream="false" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144500 2576 flags.go:64] FLAG: --logging-format="text" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144503 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144506 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144509 2576 flags.go:64] FLAG: --manifest-url="" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144512 2576 flags.go:64] FLAG: --manifest-url-header="" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144517 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144520 2576 flags.go:64] FLAG: --max-open-files="1000000" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144525 2576 flags.go:64] FLAG: --max-pods="110" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144528 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144531 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144534 2576 flags.go:64] FLAG: --memory-manager-policy="None" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144537 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144540 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144543 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144546 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144554 2576 flags.go:64] FLAG: --node-status-max-images="50" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144557 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144560 2576 flags.go:64] FLAG: --oom-score-adj="-999" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144563 2576 flags.go:64] FLAG: --pod-cidr="" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144567 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144573 2576 flags.go:64] FLAG: --pod-manifest-path="" Jun 03 15:57:07.149797 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144576 2576 flags.go:64] FLAG: --pod-max-pids="-1" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144579 2576 flags.go:64] FLAG: --pods-per-core="0" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144582 2576 flags.go:64] FLAG: --port="10250" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144585 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144588 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02cc420141db5d9a4" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144591 2576 flags.go:64] FLAG: --qos-reserved="" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144594 2576 flags.go:64] FLAG: --read-only-port="10255" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144597 2576 flags.go:64] FLAG: --register-node="true" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144602 2576 flags.go:64] FLAG: --register-schedulable="true" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144605 2576 flags.go:64] FLAG: --register-with-taints="" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144608 2576 flags.go:64] FLAG: --registry-burst="10" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144611 2576 flags.go:64] FLAG: --registry-qps="5" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144614 2576 flags.go:64] FLAG: --reserved-cpus="" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144618 2576 flags.go:64] FLAG: --reserved-memory="" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144621 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144624 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144627 2576 flags.go:64] FLAG: --rotate-certificates="false" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144630 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144633 2576 flags.go:64] FLAG: --runonce="false" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144637 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144640 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144642 2576 flags.go:64] FLAG: --seccomp-default="false" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144645 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144648 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144652 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144655 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 03 15:57:07.150405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144658 2576 flags.go:64] FLAG: --storage-driver-password="root" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144664 2576 flags.go:64] FLAG: --storage-driver-secure="false" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144667 2576 flags.go:64] FLAG: --storage-driver-table="stats" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144670 2576 flags.go:64] FLAG: --storage-driver-user="root" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144673 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144676 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144679 2576 flags.go:64] FLAG: --system-cgroups="" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144682 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144687 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144690 2576 flags.go:64] FLAG: --tls-cert-file="" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144693 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144697 2576 flags.go:64] FLAG: --tls-min-version="" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144700 2576 flags.go:64] FLAG: --tls-private-key-file="" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144703 2576 flags.go:64] FLAG: --topology-manager-policy="none" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144711 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144714 2576 flags.go:64] FLAG: --topology-manager-scope="container" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144717 2576 flags.go:64] FLAG: --v="2" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144721 2576 flags.go:64] FLAG: --version="false" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144725 2576 flags.go:64] FLAG: --vmodule="" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144730 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.144733 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144839 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144843 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144846 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144848 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 15:57:07.151051 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144851 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144854 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144857 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144860 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144864 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144866 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144869 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144871 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144875 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144878 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144881 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144883 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144886 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144889 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144893 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144897 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144900 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144902 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144905 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 15:57:07.151658 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144907 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144910 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144915 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144917 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144920 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144923 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144926 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144928 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144931 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144933 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144936 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144939 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144942 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144944 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144950 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144953 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144956 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144958 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144961 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144964 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 15:57:07.152195 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144966 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144971 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144973 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144976 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144978 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144981 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144983 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144986 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144988 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144991 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144994 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144996 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.144999 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145002 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145006 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145008 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145011 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145014 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145016 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 15:57:07.152681 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145019 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145021 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145024 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145027 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145029 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145032 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145034 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145037 2576 feature_gate.go:328] unrecognized feature gate: Example Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145040 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145043 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145045 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145048 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145050 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145053 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145058 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145061 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145064 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145067 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145069 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145072 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 15:57:07.153177 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145075 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 15:57:07.153664 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145078 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 15:57:07.153664 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145080 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 15:57:07.153664 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.145083 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 15:57:07.153664 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.145092 2576 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 15:57:07.153664 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.151785 2576 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 03 15:57:07.153664 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.151801 2576 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 03 15:57:07.153664 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151849 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 15:57:07.153664 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151855 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 15:57:07.153664 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151860 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 15:57:07.153664 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151864 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 15:57:07.153664 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151868 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 15:57:07.153664 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151871 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 15:57:07.153664 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151874 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 15:57:07.153664 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151877 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 15:57:07.153664 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151880 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151882 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151885 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151888 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151891 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151895 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151898 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151901 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151903 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151906 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151908 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151911 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151914 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151917 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151920 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151922 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151925 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151927 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151930 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151933 2576 feature_gate.go:328] unrecognized feature gate: Example Jun 03 15:57:07.154141 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151935 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151938 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151941 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151945 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151948 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151951 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151953 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151956 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151959 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151961 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151964 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151966 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151969 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151971 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151974 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151976 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151979 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151981 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151985 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151987 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 15:57:07.154617 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151990 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151993 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151995 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.151998 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152000 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152003 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152007 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152010 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152013 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152015 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152018 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152021 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152023 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152026 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152028 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152032 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152035 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152037 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152040 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152043 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 15:57:07.155130 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152046 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152048 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152051 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152053 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152056 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152058 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152061 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152064 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152066 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152069 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152071 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152074 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152077 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152080 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152082 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152085 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152088 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 15:57:07.155612 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152090 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 15:57:07.156042 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.152096 2576 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 15:57:07.156042 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152207 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 15:57:07.156042 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152212 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 15:57:07.156042 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152215 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 15:57:07.156042 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152218 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 15:57:07.156042 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152221 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 15:57:07.156042 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152224 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 15:57:07.156042 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152226 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 15:57:07.156042 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152229 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 15:57:07.156042 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152232 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 15:57:07.156042 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152235 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 15:57:07.156042 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152238 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 15:57:07.156042 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152240 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 15:57:07.156042 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152243 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 15:57:07.156042 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152246 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 15:57:07.156042 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152248 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152251 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152253 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152257 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152261 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152263 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152266 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152269 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152272 2576 feature_gate.go:328] unrecognized feature gate: Example Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152275 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152278 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152281 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152283 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152286 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152289 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152291 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152293 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152296 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152298 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 15:57:07.156440 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152301 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152303 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152306 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152308 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152311 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152313 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152316 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152319 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152322 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152325 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152327 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152330 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152332 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152335 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152337 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152339 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152342 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152345 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152347 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152350 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 15:57:07.156920 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152352 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152355 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152357 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152360 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152362 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152366 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152368 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152371 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152373 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152375 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152378 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152380 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152383 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152385 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152388 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152390 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152393 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152395 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152398 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152401 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152403 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 15:57:07.157409 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152405 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 15:57:07.157983 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152408 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 15:57:07.157983 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152411 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 15:57:07.157983 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152413 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 15:57:07.157983 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152416 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 15:57:07.157983 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152418 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 15:57:07.157983 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152422 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 15:57:07.157983 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152424 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 15:57:07.157983 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152427 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 15:57:07.157983 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152431 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 15:57:07.157983 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152433 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 15:57:07.157983 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:07.152436 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 15:57:07.157983 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.152441 2576 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 15:57:07.157983 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.153148 2576 server.go:962] "Client rotation is on, will bootstrap in background" Jun 03 15:57:07.157983 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.155187 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 03 15:57:07.157983 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.156403 2576 server.go:1019] "Starting client certificate rotation" Jun 03 15:57:07.158358 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.156500 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 15:57:07.158358 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.157101 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 15:57:07.180603 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.180582 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 15:57:07.182987 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.182962 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 15:57:07.198499 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.198474 2576 log.go:25] "Validated CRI v1 runtime API" Jun 03 15:57:07.204939 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.204923 2576 log.go:25] "Validated CRI v1 image API" Jun 03 15:57:07.206136 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.206119 2576 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 03 15:57:07.211874 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.211849 2576 fs.go:135] Filesystem UUIDs: map[0bfdd95a-cb09-45ec-b746-d0d336197f10:/dev/nvme0n1p4 464d3481-652c-466e-81a4-851bc8df92e3:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jun 03 15:57:07.211969 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.211875 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 03 15:57:07.214693 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.214673 2576 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 15:57:07.216312 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.216205 2576 manager.go:217] Machine: {Timestamp:2026-06-03 15:57:07.215704053 +0000 UTC m=+0.392673873 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3086111 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2516bfe5b3e764cd928870687d9bb8 SystemUUID:ec2516bf-e5b3-e764-cd92-8870687d9bb8 BootID:775f19d1-1001-4a35-ac88-fed9b91e9ca1 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:31:ba:4f:f5:87 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:31:ba:4f:f5:87 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:8a:6b:24:35:09 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 03 15:57:07.216973 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.216962 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 03 15:57:07.217075 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.217058 2576 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 03 15:57:07.218895 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.218869 2576 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 03 15:57:07.219062 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.218898 2576 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-61.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 03 15:57:07.219139 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.219076 2576 topology_manager.go:138] "Creating topology manager with none policy" Jun 03 15:57:07.219139 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.219088 2576 container_manager_linux.go:306] "Creating device plugin manager" Jun 03 15:57:07.219139 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.219099 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 15:57:07.219847 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.219834 2576 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 15:57:07.220749 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.220737 2576 state_mem.go:36] "Initialized new in-memory state store" Jun 03 15:57:07.220901 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.220890 2576 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 03 15:57:07.223092 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.223080 2576 kubelet.go:491] "Attempting to sync node with API server" Jun 03 15:57:07.223164 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.223108 2576 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 03 15:57:07.223164 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.223125 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 03 15:57:07.223164 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.223140 2576 kubelet.go:397] "Adding apiserver pod source" Jun 03 15:57:07.223164 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.223153 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 03 15:57:07.224378 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.224365 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 15:57:07.224439 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.224388 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 15:57:07.227376 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.227351 2576 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 03 15:57:07.229219 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.229206 2576 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 03 15:57:07.231546 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.231531 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 03 15:57:07.231546 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.231548 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 03 15:57:07.231643 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.231555 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 03 15:57:07.231643 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.231561 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 03 15:57:07.231643 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.231570 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 03 15:57:07.231643 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.231576 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 03 15:57:07.231643 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.231582 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 03 15:57:07.231643 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.231588 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 03 15:57:07.231643 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.231602 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 03 15:57:07.231643 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.231608 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 03 15:57:07.231643 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.231621 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 03 15:57:07.231643 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.231627 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 03 15:57:07.232752 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.232732 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pttzb" Jun 03 15:57:07.233124 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.233112 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 03 15:57:07.233124 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.233125 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 03 15:57:07.236114 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.236092 2576 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-61.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 03 15:57:07.236174 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.236095 2576 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 03 15:57:07.237465 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.237351 2576 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 03 15:57:07.237741 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.237728 2576 server.go:1295] "Started kubelet" Jun 03 15:57:07.238727 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.238642 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 03 15:57:07.238721 ip-10-0-128-61 systemd[1]: Started Kubernetes Kubelet. Jun 03 15:57:07.238946 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.238811 2576 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 03 15:57:07.238946 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.238880 2576 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 03 15:57:07.240169 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.240147 2576 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 03 15:57:07.241205 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.241186 2576 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-61.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 03 15:57:07.241405 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.241385 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pttzb" Jun 03 15:57:07.241750 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.241739 2576 server.go:317] "Adding debug handlers to kubelet server" Jun 03 15:57:07.248497 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.248480 2576 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 03 15:57:07.248576 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.248567 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 03 15:57:07.248639 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.248580 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 03 15:57:07.248747 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.248726 2576 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 03 15:57:07.248747 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.248744 2576 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 03 15:57:07.248940 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.248926 2576 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 03 15:57:07.249012 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.248998 2576 reconstruct.go:97] "Volume reconstruction finished" Jun 03 15:57:07.249012 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.249007 2576 reconciler.go:26] "Reconciler: start to sync state" Jun 03 15:57:07.249414 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.249394 2576 factory.go:55] Registering systemd factory Jun 03 15:57:07.249414 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.249414 2576 factory.go:223] Registration of the systemd container factory successfully Jun 03 15:57:07.249547 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.249488 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-61.ec2.internal\" not found" Jun 03 15:57:07.249638 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.249627 2576 factory.go:153] Registering CRI-O factory Jun 03 15:57:07.249677 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.249641 2576 factory.go:223] Registration of the crio container factory successfully Jun 03 15:57:07.249706 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.249686 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 03 15:57:07.249706 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.249705 2576 factory.go:103] Registering Raw factory Jun 03 15:57:07.249905 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.249720 2576 manager.go:1196] Started watching for new ooms in manager Jun 03 15:57:07.250942 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.250924 2576 manager.go:319] Starting recovery of all containers Jun 03 15:57:07.251704 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.251682 2576 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 15:57:07.255258 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.255235 2576 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-61.ec2.internal\" not found" node="ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.259301 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.259144 2576 manager.go:324] Recovery completed Jun 03 15:57:07.260808 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.260789 2576 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jun 03 15:57:07.263731 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.263719 2576 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 15:57:07.265781 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.265753 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasSufficientMemory" Jun 03 15:57:07.265869 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.265799 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 15:57:07.265869 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.265813 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasSufficientPID" Jun 03 15:57:07.266334 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.266317 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 03 15:57:07.266400 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.266334 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 03 15:57:07.266400 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.266358 2576 state_mem.go:36] "Initialized new in-memory state store" Jun 03 15:57:07.269057 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.269045 2576 policy_none.go:49] "None policy: Start" Jun 03 15:57:07.269105 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.269061 2576 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 03 15:57:07.269105 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.269072 2576 state_mem.go:35] "Initializing new in-memory state store" Jun 03 15:57:07.308416 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.308395 2576 manager.go:341] "Starting Device Plugin manager" Jun 03 15:57:07.318881 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.308428 2576 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 03 15:57:07.318881 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.308441 2576 server.go:85] "Starting device plugin registration server" Jun 03 15:57:07.318881 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.308739 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 03 15:57:07.318881 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.308751 2576 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 03 15:57:07.318881 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.308839 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 03 15:57:07.318881 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.308913 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 03 15:57:07.318881 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.308921 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 03 15:57:07.318881 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.309498 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 03 15:57:07.318881 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.309536 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-61.ec2.internal\" not found" Jun 03 15:57:07.382417 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.382377 2576 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 03 15:57:07.384214 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.384192 2576 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 03 15:57:07.384344 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.384221 2576 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 03 15:57:07.384344 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.384244 2576 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 03 15:57:07.384344 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.384252 2576 kubelet.go:2451] "Starting kubelet main sync loop" Jun 03 15:57:07.384344 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.384303 2576 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 03 15:57:07.387986 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.387970 2576 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 15:57:07.409322 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.409257 2576 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 15:57:07.410270 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.410251 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasSufficientMemory" Jun 03 15:57:07.410358 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.410285 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 15:57:07.410358 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.410296 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasSufficientPID" Jun 03 15:57:07.410358 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.410319 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.423541 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.423516 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.423541 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.423542 2576 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-61.ec2.internal\": node \"ip-10-0-128-61.ec2.internal\" not found" Jun 03 15:57:07.444885 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.444858 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-61.ec2.internal\" not found" Jun 03 15:57:07.484539 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.484500 2576 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-61.ec2.internal"] Jun 03 15:57:07.484677 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.484614 2576 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 15:57:07.487039 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.487019 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasSufficientMemory" Jun 03 15:57:07.487157 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.487054 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 15:57:07.487157 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.487068 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasSufficientPID" Jun 03 15:57:07.488506 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.488490 2576 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 15:57:07.488635 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.488618 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.488683 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.488653 2576 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 15:57:07.489317 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.489299 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasSufficientMemory" Jun 03 15:57:07.489382 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.489323 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 15:57:07.489382 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.489333 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasSufficientPID" Jun 03 15:57:07.489382 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.489304 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasSufficientMemory" Jun 03 15:57:07.489505 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.489404 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 15:57:07.489505 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.489419 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasSufficientPID" Jun 03 15:57:07.490709 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.490696 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.490752 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.490721 2576 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 15:57:07.491667 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.491652 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasSufficientMemory" Jun 03 15:57:07.491730 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.491677 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 15:57:07.491730 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.491689 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeHasSufficientPID" Jun 03 15:57:07.509443 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.509420 2576 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-61.ec2.internal\" not found" node="ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.513602 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.513585 2576 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-61.ec2.internal\" not found" node="ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.545085 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.545059 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-61.ec2.internal\" not found" Jun 03 15:57:07.550342 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.550326 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cdf3ba6c66aaec9ab5812e4869ec3138-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal\" (UID: \"cdf3ba6c66aaec9ab5812e4869ec3138\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.550403 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.550354 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cdf3ba6c66aaec9ab5812e4869ec3138-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal\" (UID: \"cdf3ba6c66aaec9ab5812e4869ec3138\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.550403 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.550373 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/790a10cb9330d251bd135e9345ddb3cd-config\") pod \"kube-apiserver-proxy-ip-10-0-128-61.ec2.internal\" (UID: \"790a10cb9330d251bd135e9345ddb3cd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.645918 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.645890 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-61.ec2.internal\" not found" Jun 03 15:57:07.651250 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.651230 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cdf3ba6c66aaec9ab5812e4869ec3138-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal\" (UID: \"cdf3ba6c66aaec9ab5812e4869ec3138\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.651314 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.651272 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cdf3ba6c66aaec9ab5812e4869ec3138-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal\" (UID: \"cdf3ba6c66aaec9ab5812e4869ec3138\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.651314 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.651277 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cdf3ba6c66aaec9ab5812e4869ec3138-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal\" (UID: \"cdf3ba6c66aaec9ab5812e4869ec3138\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.651382 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.651312 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cdf3ba6c66aaec9ab5812e4869ec3138-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal\" (UID: \"cdf3ba6c66aaec9ab5812e4869ec3138\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.651382 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.651330 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/790a10cb9330d251bd135e9345ddb3cd-config\") pod \"kube-apiserver-proxy-ip-10-0-128-61.ec2.internal\" (UID: \"790a10cb9330d251bd135e9345ddb3cd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.651382 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.651364 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/790a10cb9330d251bd135e9345ddb3cd-config\") pod \"kube-apiserver-proxy-ip-10-0-128-61.ec2.internal\" (UID: \"790a10cb9330d251bd135e9345ddb3cd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.746899 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.746835 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-61.ec2.internal\" not found" Jun 03 15:57:07.811242 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.811203 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.816850 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:07.816828 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-61.ec2.internal" Jun 03 15:57:07.847457 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.847424 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-61.ec2.internal\" not found" Jun 03 15:57:07.947959 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:07.947921 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-61.ec2.internal\" not found" Jun 03 15:57:08.048427 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:08.048351 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-61.ec2.internal\" not found" Jun 03 15:57:08.148841 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:08.148807 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-61.ec2.internal\" not found" Jun 03 15:57:08.157194 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.157169 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 03 15:57:08.157320 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.157305 2576 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 15:57:08.157359 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.157338 2576 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 15:57:08.245596 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.245557 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-02 15:52:07 +0000 UTC" deadline="2028-01-13 23:47:14.405352072 +0000 UTC" Jun 03 15:57:08.245596 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.245590 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14143h50m6.159765805s" Jun 03 15:57:08.248898 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:08.248880 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-61.ec2.internal\" not found" Jun 03 15:57:08.249003 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.248902 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 03 15:57:08.260339 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.260312 2576 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 15:57:08.279324 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.279298 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cl7b8" Jun 03 15:57:08.289670 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.289646 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cl7b8" Jun 03 15:57:08.292848 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:08.292817 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod790a10cb9330d251bd135e9345ddb3cd.slice/crio-c4fadaf84af5a91c8c7a5d281d09741d2acc04592486df9ed89ed3bbce7d0639 WatchSource:0}: Error finding container c4fadaf84af5a91c8c7a5d281d09741d2acc04592486df9ed89ed3bbce7d0639: Status 404 returned error can't find the container with id c4fadaf84af5a91c8c7a5d281d09741d2acc04592486df9ed89ed3bbce7d0639 Jun 03 15:57:08.293151 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:08.293133 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdf3ba6c66aaec9ab5812e4869ec3138.slice/crio-c70ac611444ccd56d16595e53da255e309ee050b9490cc64a433e018cae5a40e WatchSource:0}: Error finding container c70ac611444ccd56d16595e53da255e309ee050b9490cc64a433e018cae5a40e: Status 404 returned error can't find the container with id c70ac611444ccd56d16595e53da255e309ee050b9490cc64a433e018cae5a40e Jun 03 15:57:08.296452 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.296437 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 15:57:08.349627 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:08.349538 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-61.ec2.internal\" not found" Jun 03 15:57:08.387718 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.387672 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-61.ec2.internal" event={"ID":"790a10cb9330d251bd135e9345ddb3cd","Type":"ContainerStarted","Data":"c4fadaf84af5a91c8c7a5d281d09741d2acc04592486df9ed89ed3bbce7d0639"} Jun 03 15:57:08.388506 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.388478 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal" event={"ID":"cdf3ba6c66aaec9ab5812e4869ec3138","Type":"ContainerStarted","Data":"c70ac611444ccd56d16595e53da255e309ee050b9490cc64a433e018cae5a40e"} Jun 03 15:57:08.449785 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:08.449737 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-61.ec2.internal\" not found" Jun 03 15:57:08.534110 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.533971 2576 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 15:57:08.550215 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:08.550183 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-61.ec2.internal\" not found" Jun 03 15:57:08.607964 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.607882 2576 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 15:57:08.649289 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.649245 2576 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal" Jun 03 15:57:08.661062 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.661038 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 15:57:08.661909 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.661896 2576 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-61.ec2.internal" Jun 03 15:57:08.670518 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.670499 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 15:57:08.980600 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:08.980566 2576 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 15:57:09.223867 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.223835 2576 apiserver.go:52] "Watching apiserver" Jun 03 15:57:09.231104 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.231037 2576 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 03 15:57:09.233892 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.233864 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal","openshift-multus/multus-tdzqn","openshift-network-operator/iptables-alerter-z289q","openshift-ovn-kubernetes/ovnkube-node-bjv7n","openshift-cluster-node-tuning-operator/tuned-p4fdl","openshift-dns/node-resolver-q5cmz","openshift-image-registry/node-ca-4ppxs","openshift-multus/multus-additional-cni-plugins-j7xtj","openshift-multus/network-metrics-daemon-w5jsj","openshift-network-diagnostics/network-check-target-g74tp","kube-system/konnectivity-agent-7w5ch","kube-system/kube-apiserver-proxy-ip-10-0-128-61.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr"] Jun 03 15:57:09.236692 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.236667 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4ppxs" Jun 03 15:57:09.238019 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.237996 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.239262 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.239242 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z289q" Jun 03 15:57:09.239487 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.239399 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.240566 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.240542 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.241892 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.241874 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q5cmz" Jun 03 15:57:09.242100 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.242075 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2pwg8\"" Jun 03 15:57:09.242295 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.242273 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 03 15:57:09.242624 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.242379 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 03 15:57:09.242624 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.242410 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 03 15:57:09.242624 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.242519 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 03 15:57:09.242624 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.242542 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hw7js\"" Jun 03 15:57:09.242624 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.242561 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 03 15:57:09.242624 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.242593 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 03 15:57:09.243066 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.242679 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 03 15:57:09.243066 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.242704 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 03 15:57:09.243066 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.242721 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 03 15:57:09.243066 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.242736 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 03 15:57:09.243066 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.242820 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 03 15:57:09.243066 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.243028 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 03 15:57:09.243386 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.243313 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 03 15:57:09.243386 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.243324 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bs6tp\"" Jun 03 15:57:09.243527 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.243415 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k276t\"" Jun 03 15:57:09.243605 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.243538 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 03 15:57:09.243605 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.243547 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 03 15:57:09.243905 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.243859 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nth2g\"" Jun 03 15:57:09.243905 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.243896 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 03 15:57:09.244098 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.244057 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 03 15:57:09.244156 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.244124 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 03 15:57:09.244156 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.244133 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 03 15:57:09.244294 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.244273 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 03 15:57:09.244294 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.244274 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-j92gh\"" Jun 03 15:57:09.245433 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.245412 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:09.245702 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:09.245664 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:09.246227 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.246038 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.248853 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.248830 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 03 15:57:09.249098 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.249079 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m2pmc\"" Jun 03 15:57:09.249204 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.249141 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 03 15:57:09.249951 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.249933 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:09.250108 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:09.250086 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:09.251598 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.251576 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7w5ch" Jun 03 15:57:09.253726 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.253705 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 03 15:57:09.253841 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.253736 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 03 15:57:09.253841 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.253764 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gfs7v\"" Jun 03 15:57:09.253981 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.253963 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.254858 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.254839 2576 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 15:57:09.255995 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.255979 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 03 15:57:09.256087 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.256006 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 03 15:57:09.256284 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.256267 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 03 15:57:09.256349 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.256271 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5l67q\"" Jun 03 15:57:09.259473 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259441 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xdvs\" (UniqueName: \"kubernetes.io/projected/16c438af-88c9-4b06-be30-c984a697afeb-kube-api-access-9xdvs\") pod \"node-resolver-q5cmz\" (UID: \"16c438af-88c9-4b06-be30-c984a697afeb\") " pod="openshift-dns/node-resolver-q5cmz" Jun 03 15:57:09.259555 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259487 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-var-lib-openvswitch\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.259555 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259514 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs\") pod \"network-metrics-daemon-w5jsj\" (UID: \"2b455f74-b695-4158-8fe3-2fe335bd3e3b\") " pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:09.259555 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259541 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-cnibin\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.259814 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259565 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31941200-bc35-48cc-9148-b104dbfde42e-cni-binary-copy\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.259814 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259588 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-run-systemd\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.259814 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259611 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-etc-openvswitch\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.259814 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259636 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/27386c54-e43c-47d4-89e2-7ae61fb82e65-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.259814 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259665 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-system-cni-dir\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.259814 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259699 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-modprobe-d\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.259814 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259726 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-sysctl-d\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.259814 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259755 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-host\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.259814 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259798 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e1d606f-ce5b-4871-b678-8d93934e06bf-tmp\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.260223 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259850 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-multus-socket-dir-parent\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.260223 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259874 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-run-netns\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.260223 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259899 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-var-lib-kubelet\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.260223 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259923 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-node-log\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.260223 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259947 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-multus-cni-dir\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.260223 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.259982 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-hostroot\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.260223 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260007 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-var-lib-kubelet\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.260223 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260061 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16c438af-88c9-4b06-be30-c984a697afeb-tmp-dir\") pod \"node-resolver-q5cmz\" (UID: \"16c438af-88c9-4b06-be30-c984a697afeb\") " pod="openshift-dns/node-resolver-q5cmz" Jun 03 15:57:09.260223 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260090 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-systemd-units\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.260223 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260115 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-log-socket\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.260223 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260137 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6m48\" (UniqueName: \"kubernetes.io/projected/490f0a30-89ae-4e18-87f0-498a714ff08c-kube-api-access-b6m48\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.260223 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260162 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r45hb\" (UniqueName: \"kubernetes.io/projected/2b455f74-b695-4158-8fe3-2fe335bd3e3b-kube-api-access-r45hb\") pod \"network-metrics-daemon-w5jsj\" (UID: \"2b455f74-b695-4158-8fe3-2fe335bd3e3b\") " pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:09.260223 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260186 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-multus-conf-dir\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.260223 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260213 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/27386c54-e43c-47d4-89e2-7ae61fb82e65-os-release\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.260860 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260239 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/27386c54-e43c-47d4-89e2-7ae61fb82e65-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.260860 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260267 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl2cd\" (UniqueName: \"kubernetes.io/projected/0aa350a0-203c-4286-b184-d62efbd0af97-kube-api-access-bl2cd\") pod \"node-ca-4ppxs\" (UID: \"0aa350a0-203c-4286-b184-d62efbd0af97\") " pod="openshift-image-registry/node-ca-4ppxs" Jun 03 15:57:09.260860 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260292 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-kubernetes\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.260860 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260373 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-run-openvswitch\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.260860 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260433 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-run-ovn\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.260860 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260478 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/490f0a30-89ae-4e18-87f0-498a714ff08c-ovnkube-script-lib\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.260860 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260538 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-lib-modules\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.260860 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260583 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-kubelet\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.260860 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260623 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/27386c54-e43c-47d4-89e2-7ae61fb82e65-cnibin\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.260860 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260662 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/27386c54-e43c-47d4-89e2-7ae61fb82e65-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.260860 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260693 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0aa350a0-203c-4286-b184-d62efbd0af97-host\") pod \"node-ca-4ppxs\" (UID: \"0aa350a0-203c-4286-b184-d62efbd0af97\") " pod="openshift-image-registry/node-ca-4ppxs" Jun 03 15:57:09.260860 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260717 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0aa350a0-203c-4286-b184-d62efbd0af97-serviceca\") pod \"node-ca-4ppxs\" (UID: \"0aa350a0-203c-4286-b184-d62efbd0af97\") " pod="openshift-image-registry/node-ca-4ppxs" Jun 03 15:57:09.260860 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260741 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/31941200-bc35-48cc-9148-b104dbfde42e-multus-daemon-config\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.260860 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260795 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg569\" (UniqueName: \"kubernetes.io/projected/c1c44950-8787-4b29-9f1b-c6a191f81431-kube-api-access-tg569\") pod \"iptables-alerter-z289q\" (UID: \"c1c44950-8787-4b29-9f1b-c6a191f81431\") " pod="openshift-network-operator/iptables-alerter-z289q" Jun 03 15:57:09.260860 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260819 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn4zm\" (UniqueName: \"kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm\") pod \"network-check-target-g74tp\" (UID: \"5bcd65d8-45ad-42e2-8554-16e218c5b465\") " pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:09.260860 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260852 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/27386c54-e43c-47d4-89e2-7ae61fb82e65-system-cni-dir\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260880 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-run\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260923 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-sys\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260948 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-run-ovn-kubernetes\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.260972 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-cni-bin\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261004 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1c44950-8787-4b29-9f1b-c6a191f81431-host-slash\") pod \"iptables-alerter-z289q\" (UID: \"c1c44950-8787-4b29-9f1b-c6a191f81431\") " pod="openshift-network-operator/iptables-alerter-z289q" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261028 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261051 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/490f0a30-89ae-4e18-87f0-498a714ff08c-env-overrides\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261073 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-var-lib-cni-bin\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261099 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/490f0a30-89ae-4e18-87f0-498a714ff08c-ovnkube-config\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261124 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-run-multus-certs\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261147 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-etc-kubernetes\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261200 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-sysconfig\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261234 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/16c438af-88c9-4b06-be30-c984a697afeb-hosts-file\") pod \"node-resolver-q5cmz\" (UID: \"16c438af-88c9-4b06-be30-c984a697afeb\") " pod="openshift-dns/node-resolver-q5cmz" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261291 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-slash\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261345 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-run-netns\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261374 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvffj\" (UniqueName: \"kubernetes.io/projected/31941200-bc35-48cc-9148-b104dbfde42e-kube-api-access-zvffj\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.261551 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261399 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c1c44950-8787-4b29-9f1b-c6a191f81431-iptables-alerter-script\") pod \"iptables-alerter-z289q\" (UID: \"c1c44950-8787-4b29-9f1b-c6a191f81431\") " pod="openshift-network-operator/iptables-alerter-z289q" Jun 03 15:57:09.262214 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261416 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-systemd\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.262214 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261435 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-cni-netd\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.262214 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261455 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj6lv\" (UniqueName: \"kubernetes.io/projected/27386c54-e43c-47d4-89e2-7ae61fb82e65-kube-api-access-bj6lv\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.262214 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261478 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-run-k8s-cni-cncf-io\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.262214 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261492 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-var-lib-cni-multus\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.262214 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261513 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-sysctl-conf\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.262214 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261535 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd6dg\" (UniqueName: \"kubernetes.io/projected/7e1d606f-ce5b-4871-b678-8d93934e06bf-kube-api-access-fd6dg\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.262214 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261585 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/490f0a30-89ae-4e18-87f0-498a714ff08c-ovn-node-metrics-cert\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.262214 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261620 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/27386c54-e43c-47d4-89e2-7ae61fb82e65-cni-binary-copy\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.262214 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261644 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-os-release\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.262214 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.261668 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-tuned\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.290853 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.290821 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 15:52:08 +0000 UTC" deadline="2027-11-03 12:45:57.291755233 +0000 UTC" Jun 03 15:57:09.290962 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.290854 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12428h48m48.000907429s" Jun 03 15:57:09.349962 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.349927 2576 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 03 15:57:09.361953 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.361918 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-system-cni-dir\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.361953 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.361959 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-modprobe-d\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.362188 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362020 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-sysctl-d\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.362188 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362043 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-system-cni-dir\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.362188 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362058 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-host\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.362188 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362079 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-modprobe-d\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.362188 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362085 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e1d606f-ce5b-4871-b678-8d93934e06bf-tmp\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.362188 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362143 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-multus-socket-dir-parent\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.362188 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362174 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-run-netns\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.362188 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362178 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-sysctl-d\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.362188 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362144 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-host\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.362188 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362196 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-var-lib-kubelet\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362219 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-node-log\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362235 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-run-netns\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362248 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-multus-cni-dir\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362256 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-var-lib-kubelet\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362272 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-hostroot\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362285 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-multus-socket-dir-parent\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362291 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-node-log\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362295 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-var-lib-kubelet\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362321 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-multus-cni-dir\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362349 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-var-lib-kubelet\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362359 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-hostroot\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362385 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16c438af-88c9-4b06-be30-c984a697afeb-tmp-dir\") pod \"node-resolver-q5cmz\" (UID: \"16c438af-88c9-4b06-be30-c984a697afeb\") " pod="openshift-dns/node-resolver-q5cmz" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362425 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-systemd-units\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362449 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-log-socket\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362473 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b6m48\" (UniqueName: \"kubernetes.io/projected/490f0a30-89ae-4e18-87f0-498a714ff08c-kube-api-access-b6m48\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362481 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362497 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r45hb\" (UniqueName: \"kubernetes.io/projected/2b455f74-b695-4158-8fe3-2fe335bd3e3b-kube-api-access-r45hb\") pod \"network-metrics-daemon-w5jsj\" (UID: \"2b455f74-b695-4158-8fe3-2fe335bd3e3b\") " pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:09.362594 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362519 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-multus-conf-dir\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362524 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-log-socket\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362542 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/27386c54-e43c-47d4-89e2-7ae61fb82e65-os-release\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362569 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-multus-conf-dir\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362571 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/27386c54-e43c-47d4-89e2-7ae61fb82e65-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362487 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-systemd-units\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362611 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/128d979f-9fbc-4899-8f0a-b0023e619799-konnectivity-ca\") pod \"konnectivity-agent-7w5ch\" (UID: \"128d979f-9fbc-4899-8f0a-b0023e619799\") " pod="kube-system/konnectivity-agent-7w5ch" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362638 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bl2cd\" (UniqueName: \"kubernetes.io/projected/0aa350a0-203c-4286-b184-d62efbd0af97-kube-api-access-bl2cd\") pod \"node-ca-4ppxs\" (UID: \"0aa350a0-203c-4286-b184-d62efbd0af97\") " pod="openshift-image-registry/node-ca-4ppxs" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362656 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/27386c54-e43c-47d4-89e2-7ae61fb82e65-os-release\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362662 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-kubernetes\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362685 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16c438af-88c9-4b06-be30-c984a697afeb-tmp-dir\") pod \"node-resolver-q5cmz\" (UID: \"16c438af-88c9-4b06-be30-c984a697afeb\") " pod="openshift-dns/node-resolver-q5cmz" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362695 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-run-openvswitch\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362706 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-kubernetes\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362725 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-run-ovn\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362749 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-run-openvswitch\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362750 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/490f0a30-89ae-4e18-87f0-498a714ff08c-ovnkube-script-lib\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362803 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-run-ovn\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362809 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-lib-modules\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.363360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362848 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-kubelet\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362889 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/27386c54-e43c-47d4-89e2-7ae61fb82e65-cnibin\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362915 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/27386c54-e43c-47d4-89e2-7ae61fb82e65-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362923 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-lib-modules\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362921 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-kubelet\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362951 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6c5t\" (UniqueName: \"kubernetes.io/projected/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-kube-api-access-h6c5t\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362975 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0aa350a0-203c-4286-b184-d62efbd0af97-host\") pod \"node-ca-4ppxs\" (UID: \"0aa350a0-203c-4286-b184-d62efbd0af97\") " pod="openshift-image-registry/node-ca-4ppxs" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362975 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/27386c54-e43c-47d4-89e2-7ae61fb82e65-cnibin\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.362996 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0aa350a0-203c-4286-b184-d62efbd0af97-serviceca\") pod \"node-ca-4ppxs\" (UID: \"0aa350a0-203c-4286-b184-d62efbd0af97\") " pod="openshift-image-registry/node-ca-4ppxs" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363017 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/31941200-bc35-48cc-9148-b104dbfde42e-multus-daemon-config\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363038 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tg569\" (UniqueName: \"kubernetes.io/projected/c1c44950-8787-4b29-9f1b-c6a191f81431-kube-api-access-tg569\") pod \"iptables-alerter-z289q\" (UID: \"c1c44950-8787-4b29-9f1b-c6a191f81431\") " pod="openshift-network-operator/iptables-alerter-z289q" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363048 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0aa350a0-203c-4286-b184-d62efbd0af97-host\") pod \"node-ca-4ppxs\" (UID: \"0aa350a0-203c-4286-b184-d62efbd0af97\") " pod="openshift-image-registry/node-ca-4ppxs" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363054 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/27386c54-e43c-47d4-89e2-7ae61fb82e65-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363069 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hn4zm\" (UniqueName: \"kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm\") pod \"network-check-target-g74tp\" (UID: \"5bcd65d8-45ad-42e2-8554-16e218c5b465\") " pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363097 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/27386c54-e43c-47d4-89e2-7ae61fb82e65-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363108 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/27386c54-e43c-47d4-89e2-7ae61fb82e65-system-cni-dir\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363132 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-socket-dir\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.364186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363155 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-run\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363175 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-sys\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363194 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-sys-fs\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363216 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-run-ovn-kubernetes\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363238 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-cni-bin\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363259 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/128d979f-9fbc-4899-8f0a-b0023e619799-agent-certs\") pod \"konnectivity-agent-7w5ch\" (UID: \"128d979f-9fbc-4899-8f0a-b0023e619799\") " pod="kube-system/konnectivity-agent-7w5ch" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363286 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1c44950-8787-4b29-9f1b-c6a191f81431-host-slash\") pod \"iptables-alerter-z289q\" (UID: \"c1c44950-8787-4b29-9f1b-c6a191f81431\") " pod="openshift-network-operator/iptables-alerter-z289q" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363303 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/27386c54-e43c-47d4-89e2-7ae61fb82e65-system-cni-dir\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363309 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363332 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/490f0a30-89ae-4e18-87f0-498a714ff08c-env-overrides\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363370 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-run\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363371 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/490f0a30-89ae-4e18-87f0-498a714ff08c-ovnkube-script-lib\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363371 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-device-dir\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363418 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-var-lib-cni-bin\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363439 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-cni-bin\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363446 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/490f0a30-89ae-4e18-87f0-498a714ff08c-ovnkube-config\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363455 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0aa350a0-203c-4286-b184-d62efbd0af97-serviceca\") pod \"node-ca-4ppxs\" (UID: \"0aa350a0-203c-4286-b184-d62efbd0af97\") " pod="openshift-image-registry/node-ca-4ppxs" Jun 03 15:57:09.364898 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363418 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-run-ovn-kubernetes\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363461 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-var-lib-cni-bin\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363477 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-registration-dir\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363489 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363518 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-run-multus-certs\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363524 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1c44950-8787-4b29-9f1b-c6a191f81431-host-slash\") pod \"iptables-alerter-z289q\" (UID: \"c1c44950-8787-4b29-9f1b-c6a191f81431\") " pod="openshift-network-operator/iptables-alerter-z289q" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363557 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-sys\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363573 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-run-multus-certs\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363573 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/31941200-bc35-48cc-9148-b104dbfde42e-multus-daemon-config\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363618 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-etc-kubernetes\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363645 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-sysconfig\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363671 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/16c438af-88c9-4b06-be30-c984a697afeb-hosts-file\") pod \"node-resolver-q5cmz\" (UID: \"16c438af-88c9-4b06-be30-c984a697afeb\") " pod="openshift-dns/node-resolver-q5cmz" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363674 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-etc-kubernetes\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363696 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-slash\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363702 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/490f0a30-89ae-4e18-87f0-498a714ff08c-env-overrides\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363728 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-sysconfig\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363738 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-slash\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363786 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-run-netns\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.365592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363807 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-run-netns\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363751 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/16c438af-88c9-4b06-be30-c984a697afeb-hosts-file\") pod \"node-resolver-q5cmz\" (UID: \"16c438af-88c9-4b06-be30-c984a697afeb\") " pod="openshift-dns/node-resolver-q5cmz" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363832 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zvffj\" (UniqueName: \"kubernetes.io/projected/31941200-bc35-48cc-9148-b104dbfde42e-kube-api-access-zvffj\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363858 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c1c44950-8787-4b29-9f1b-c6a191f81431-iptables-alerter-script\") pod \"iptables-alerter-z289q\" (UID: \"c1c44950-8787-4b29-9f1b-c6a191f81431\") " pod="openshift-network-operator/iptables-alerter-z289q" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.363999 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/490f0a30-89ae-4e18-87f0-498a714ff08c-ovnkube-config\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364097 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-systemd\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364126 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-cni-netd\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364151 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bj6lv\" (UniqueName: \"kubernetes.io/projected/27386c54-e43c-47d4-89e2-7ae61fb82e65-kube-api-access-bj6lv\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364184 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364205 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-systemd\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364196 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-host-cni-netd\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364208 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-run-k8s-cni-cncf-io\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364247 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-run-k8s-cni-cncf-io\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364266 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-var-lib-cni-multus\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364292 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-sysctl-conf\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364317 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fd6dg\" (UniqueName: \"kubernetes.io/projected/7e1d606f-ce5b-4871-b678-8d93934e06bf-kube-api-access-fd6dg\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364330 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-host-var-lib-cni-multus\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.366316 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364340 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/490f0a30-89ae-4e18-87f0-498a714ff08c-ovn-node-metrics-cert\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364379 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/27386c54-e43c-47d4-89e2-7ae61fb82e65-cni-binary-copy\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364445 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-sysctl-conf\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364461 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c1c44950-8787-4b29-9f1b-c6a191f81431-iptables-alerter-script\") pod \"iptables-alerter-z289q\" (UID: \"c1c44950-8787-4b29-9f1b-c6a191f81431\") " pod="openshift-network-operator/iptables-alerter-z289q" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364694 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-etc-selinux\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364734 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-os-release\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364764 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-tuned\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364807 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/27386c54-e43c-47d4-89e2-7ae61fb82e65-cni-binary-copy\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364813 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9xdvs\" (UniqueName: \"kubernetes.io/projected/16c438af-88c9-4b06-be30-c984a697afeb-kube-api-access-9xdvs\") pod \"node-resolver-q5cmz\" (UID: \"16c438af-88c9-4b06-be30-c984a697afeb\") " pod="openshift-dns/node-resolver-q5cmz" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364839 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-var-lib-openvswitch\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364863 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs\") pod \"network-metrics-daemon-w5jsj\" (UID: \"2b455f74-b695-4158-8fe3-2fe335bd3e3b\") " pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364889 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-cnibin\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364913 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31941200-bc35-48cc-9148-b104dbfde42e-cni-binary-copy\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364939 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-run-systemd\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.364974 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-etc-openvswitch\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.365000 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/27386c54-e43c-47d4-89e2-7ae61fb82e65-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.365066 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-cnibin\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.365228 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-var-lib-openvswitch\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.366993 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:09.365279 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:57:09.367803 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.365316 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-etc-openvswitch\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.367803 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.365282 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/31941200-bc35-48cc-9148-b104dbfde42e-os-release\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.367803 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.365282 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/490f0a30-89ae-4e18-87f0-498a714ff08c-run-systemd\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.367803 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:09.365411 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs podName:2b455f74-b695-4158-8fe3-2fe335bd3e3b nodeName:}" failed. No retries permitted until 2026-06-03 15:57:09.865390781 +0000 UTC m=+3.042360591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs") pod "network-metrics-daemon-w5jsj" (UID: "2b455f74-b695-4158-8fe3-2fe335bd3e3b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:57:09.367803 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.365715 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/27386c54-e43c-47d4-89e2-7ae61fb82e65-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.367803 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.365726 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/31941200-bc35-48cc-9148-b104dbfde42e-cni-binary-copy\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.367803 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.366350 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e1d606f-ce5b-4871-b678-8d93934e06bf-tmp\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.367803 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.367009 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7e1d606f-ce5b-4871-b678-8d93934e06bf-etc-tuned\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.367803 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.367376 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/490f0a30-89ae-4e18-87f0-498a714ff08c-ovn-node-metrics-cert\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.368851 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:09.368830 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 15:57:09.368851 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:09.368851 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 15:57:09.369001 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:09.368861 2576 projected.go:194] Error preparing data for projected volume kube-api-access-hn4zm for pod openshift-network-diagnostics/network-check-target-g74tp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:57:09.369001 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:09.368927 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm podName:5bcd65d8-45ad-42e2-8554-16e218c5b465 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:09.868908595 +0000 UTC m=+3.045878382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hn4zm" (UniqueName: "kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm") pod "network-check-target-g74tp" (UID: "5bcd65d8-45ad-42e2-8554-16e218c5b465") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:57:09.371554 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.371401 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6m48\" (UniqueName: \"kubernetes.io/projected/490f0a30-89ae-4e18-87f0-498a714ff08c-kube-api-access-b6m48\") pod \"ovnkube-node-bjv7n\" (UID: \"490f0a30-89ae-4e18-87f0-498a714ff08c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.371554 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.371516 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl2cd\" (UniqueName: \"kubernetes.io/projected/0aa350a0-203c-4286-b184-d62efbd0af97-kube-api-access-bl2cd\") pod \"node-ca-4ppxs\" (UID: \"0aa350a0-203c-4286-b184-d62efbd0af97\") " pod="openshift-image-registry/node-ca-4ppxs" Jun 03 15:57:09.371702 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.371615 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r45hb\" (UniqueName: \"kubernetes.io/projected/2b455f74-b695-4158-8fe3-2fe335bd3e3b-kube-api-access-r45hb\") pod \"network-metrics-daemon-w5jsj\" (UID: \"2b455f74-b695-4158-8fe3-2fe335bd3e3b\") " pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:09.372248 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.372222 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg569\" (UniqueName: \"kubernetes.io/projected/c1c44950-8787-4b29-9f1b-c6a191f81431-kube-api-access-tg569\") pod \"iptables-alerter-z289q\" (UID: \"c1c44950-8787-4b29-9f1b-c6a191f81431\") " pod="openshift-network-operator/iptables-alerter-z289q" Jun 03 15:57:09.372840 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.372820 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd6dg\" (UniqueName: \"kubernetes.io/projected/7e1d606f-ce5b-4871-b678-8d93934e06bf-kube-api-access-fd6dg\") pod \"tuned-p4fdl\" (UID: \"7e1d606f-ce5b-4871-b678-8d93934e06bf\") " pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.373903 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.373840 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvffj\" (UniqueName: \"kubernetes.io/projected/31941200-bc35-48cc-9148-b104dbfde42e-kube-api-access-zvffj\") pod \"multus-tdzqn\" (UID: \"31941200-bc35-48cc-9148-b104dbfde42e\") " pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.374031 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.373929 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj6lv\" (UniqueName: \"kubernetes.io/projected/27386c54-e43c-47d4-89e2-7ae61fb82e65-kube-api-access-bj6lv\") pod \"multus-additional-cni-plugins-j7xtj\" (UID: \"27386c54-e43c-47d4-89e2-7ae61fb82e65\") " pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.374547 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.374530 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xdvs\" (UniqueName: \"kubernetes.io/projected/16c438af-88c9-4b06-be30-c984a697afeb-kube-api-access-9xdvs\") pod \"node-resolver-q5cmz\" (UID: \"16c438af-88c9-4b06-be30-c984a697afeb\") " pod="openshift-dns/node-resolver-q5cmz" Jun 03 15:57:09.465606 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.465576 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/128d979f-9fbc-4899-8f0a-b0023e619799-konnectivity-ca\") pod \"konnectivity-agent-7w5ch\" (UID: \"128d979f-9fbc-4899-8f0a-b0023e619799\") " pod="kube-system/konnectivity-agent-7w5ch" Jun 03 15:57:09.465606 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.465618 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h6c5t\" (UniqueName: \"kubernetes.io/projected/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-kube-api-access-h6c5t\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.465843 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.465656 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-socket-dir\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.465843 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.465677 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-sys-fs\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.465843 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.465703 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/128d979f-9fbc-4899-8f0a-b0023e619799-agent-certs\") pod \"konnectivity-agent-7w5ch\" (UID: \"128d979f-9fbc-4899-8f0a-b0023e619799\") " pod="kube-system/konnectivity-agent-7w5ch" Jun 03 15:57:09.465843 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.465820 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-sys-fs\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.465980 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.465864 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-socket-dir\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.465980 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.465942 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-device-dir\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.466049 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.465976 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-registration-dir\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.466049 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.466012 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.466049 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.466034 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-device-dir\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.466049 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.466044 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-etc-selinux\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.466174 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.466067 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-registration-dir\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.466174 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.466094 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.466259 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.466176 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-etc-selinux\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.466259 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.466221 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/128d979f-9fbc-4899-8f0a-b0023e619799-konnectivity-ca\") pod \"konnectivity-agent-7w5ch\" (UID: \"128d979f-9fbc-4899-8f0a-b0023e619799\") " pod="kube-system/konnectivity-agent-7w5ch" Jun 03 15:57:09.469226 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.469190 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/128d979f-9fbc-4899-8f0a-b0023e619799-agent-certs\") pod \"konnectivity-agent-7w5ch\" (UID: \"128d979f-9fbc-4899-8f0a-b0023e619799\") " pod="kube-system/konnectivity-agent-7w5ch" Jun 03 15:57:09.474473 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.474449 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6c5t\" (UniqueName: \"kubernetes.io/projected/2544ae4d-5cee-47ff-b8db-2e27aefebf5c-kube-api-access-h6c5t\") pod \"aws-ebs-csi-driver-node-dm5xr\" (UID: \"2544ae4d-5cee-47ff-b8db-2e27aefebf5c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.549401 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.549315 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4ppxs" Jun 03 15:57:09.556433 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.556400 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tdzqn" Jun 03 15:57:09.565717 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.565690 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z289q" Jun 03 15:57:09.571480 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.571456 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:09.578154 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.578133 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" Jun 03 15:57:09.583617 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.583595 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q5cmz" Jun 03 15:57:09.590187 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.590151 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j7xtj" Jun 03 15:57:09.597889 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.597866 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7w5ch" Jun 03 15:57:09.602547 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.602525 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" Jun 03 15:57:09.868273 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.868191 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs\") pod \"network-metrics-daemon-w5jsj\" (UID: \"2b455f74-b695-4158-8fe3-2fe335bd3e3b\") " pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:09.868437 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:09.868314 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:57:09.868437 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:09.868382 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs podName:2b455f74-b695-4158-8fe3-2fe335bd3e3b nodeName:}" failed. No retries permitted until 2026-06-03 15:57:10.868364886 +0000 UTC m=+4.045334673 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs") pod "network-metrics-daemon-w5jsj" (UID: "2b455f74-b695-4158-8fe3-2fe335bd3e3b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:57:09.968581 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:09.968544 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hn4zm\" (UniqueName: \"kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm\") pod \"network-check-target-g74tp\" (UID: \"5bcd65d8-45ad-42e2-8554-16e218c5b465\") " pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:09.968815 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:09.968722 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 15:57:09.968815 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:09.968745 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 15:57:09.968815 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:09.968759 2576 projected.go:194] Error preparing data for projected volume kube-api-access-hn4zm for pod openshift-network-diagnostics/network-check-target-g74tp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:57:09.968966 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:09.968833 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm podName:5bcd65d8-45ad-42e2-8554-16e218c5b465 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:10.968814529 +0000 UTC m=+4.145784317 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hn4zm" (UniqueName: "kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm") pod "network-check-target-g74tp" (UID: "5bcd65d8-45ad-42e2-8554-16e218c5b465") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:57:09.973220 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:09.973195 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod490f0a30_89ae_4e18_87f0_498a714ff08c.slice/crio-10194eb1dfaaf00cb88047cb840d68a1195ce7dd5d1d1cec4788eea8e489a18a WatchSource:0}: Error finding container 10194eb1dfaaf00cb88047cb840d68a1195ce7dd5d1d1cec4788eea8e489a18a: Status 404 returned error can't find the container with id 10194eb1dfaaf00cb88047cb840d68a1195ce7dd5d1d1cec4788eea8e489a18a Jun 03 15:57:09.974997 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:09.974971 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa350a0_203c_4286_b184_d62efbd0af97.slice/crio-feb3302c3386fc149156d8244cb3af196d65c40b498c6b8dad4d505617e359dd WatchSource:0}: Error finding container feb3302c3386fc149156d8244cb3af196d65c40b498c6b8dad4d505617e359dd: Status 404 returned error can't find the container with id feb3302c3386fc149156d8244cb3af196d65c40b498c6b8dad4d505617e359dd Jun 03 15:57:09.976067 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:09.976048 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod128d979f_9fbc_4899_8f0a_b0023e619799.slice/crio-474b0def2106d95532369bc4df7df3c6cffdcad9e98c20be9d55a3bf680b2c97 WatchSource:0}: Error finding container 474b0def2106d95532369bc4df7df3c6cffdcad9e98c20be9d55a3bf680b2c97: Status 404 returned error can't find the container with id 474b0def2106d95532369bc4df7df3c6cffdcad9e98c20be9d55a3bf680b2c97 Jun 03 15:57:09.978245 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:09.978147 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16c438af_88c9_4b06_be30_c984a697afeb.slice/crio-eaabc3ddc07573e89d3fa852f3cc73cbe0e6a58540d70f91af945f76e26498c2 WatchSource:0}: Error finding container eaabc3ddc07573e89d3fa852f3cc73cbe0e6a58540d70f91af945f76e26498c2: Status 404 returned error can't find the container with id eaabc3ddc07573e89d3fa852f3cc73cbe0e6a58540d70f91af945f76e26498c2 Jun 03 15:57:09.978831 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:09.978812 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e1d606f_ce5b_4871_b678_8d93934e06bf.slice/crio-20e7c7e8a4c700832412d192c0bff872e969be73fafe15d9ef10a251cfe51b4a WatchSource:0}: Error finding container 20e7c7e8a4c700832412d192c0bff872e969be73fafe15d9ef10a251cfe51b4a: Status 404 returned error can't find the container with id 20e7c7e8a4c700832412d192c0bff872e969be73fafe15d9ef10a251cfe51b4a Jun 03 15:57:09.980882 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:09.980860 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27386c54_e43c_47d4_89e2_7ae61fb82e65.slice/crio-84a73c450d3608d15606272d64fc0d790f9d67ccff8ca1409617ce1edbd082f1 WatchSource:0}: Error finding container 84a73c450d3608d15606272d64fc0d790f9d67ccff8ca1409617ce1edbd082f1: Status 404 returned error can't find the container with id 84a73c450d3608d15606272d64fc0d790f9d67ccff8ca1409617ce1edbd082f1 Jun 03 15:57:09.981342 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:09.981322 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2544ae4d_5cee_47ff_b8db_2e27aefebf5c.slice/crio-7f07a9661b1b1b08e5f33b6f2262468158fb3f6dbeefc946d0dbbe7305cca24e WatchSource:0}: Error finding container 7f07a9661b1b1b08e5f33b6f2262468158fb3f6dbeefc946d0dbbe7305cca24e: Status 404 returned error can't find the container with id 7f07a9661b1b1b08e5f33b6f2262468158fb3f6dbeefc946d0dbbe7305cca24e Jun 03 15:57:09.982699 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:09.982675 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31941200_bc35_48cc_9148_b104dbfde42e.slice/crio-5fc41f5f30f03b4c29e2a7139b1719cb89920766c9482f5dfd6b91ae922a6a1d WatchSource:0}: Error finding container 5fc41f5f30f03b4c29e2a7139b1719cb89920766c9482f5dfd6b91ae922a6a1d: Status 404 returned error can't find the container with id 5fc41f5f30f03b4c29e2a7139b1719cb89920766c9482f5dfd6b91ae922a6a1d Jun 03 15:57:09.983935 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:09.983914 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1c44950_8787_4b29_9f1b_c6a191f81431.slice/crio-ed5803fc00aca3aee112b65f84134e52f4a02e83dbfa931b3b562ff11393c50e WatchSource:0}: Error finding container ed5803fc00aca3aee112b65f84134e52f4a02e83dbfa931b3b562ff11393c50e: Status 404 returned error can't find the container with id ed5803fc00aca3aee112b65f84134e52f4a02e83dbfa931b3b562ff11393c50e Jun 03 15:57:10.292073 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.291880 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 15:52:08 +0000 UTC" deadline="2027-11-18 14:38:46.867851975 +0000 UTC" Jun 03 15:57:10.292073 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.292069 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12790h41m36.57578641s" Jun 03 15:57:10.393893 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.393856 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" event={"ID":"2544ae4d-5cee-47ff-b8db-2e27aefebf5c","Type":"ContainerStarted","Data":"7f07a9661b1b1b08e5f33b6f2262468158fb3f6dbeefc946d0dbbe7305cca24e"} Jun 03 15:57:10.396457 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.396384 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7xtj" event={"ID":"27386c54-e43c-47d4-89e2-7ae61fb82e65","Type":"ContainerStarted","Data":"84a73c450d3608d15606272d64fc0d790f9d67ccff8ca1409617ce1edbd082f1"} Jun 03 15:57:10.398962 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.398888 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" event={"ID":"7e1d606f-ce5b-4871-b678-8d93934e06bf","Type":"ContainerStarted","Data":"20e7c7e8a4c700832412d192c0bff872e969be73fafe15d9ef10a251cfe51b4a"} Jun 03 15:57:10.404323 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.404122 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q5cmz" event={"ID":"16c438af-88c9-4b06-be30-c984a697afeb","Type":"ContainerStarted","Data":"eaabc3ddc07573e89d3fa852f3cc73cbe0e6a58540d70f91af945f76e26498c2"} Jun 03 15:57:10.409609 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.409542 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7w5ch" event={"ID":"128d979f-9fbc-4899-8f0a-b0023e619799","Type":"ContainerStarted","Data":"474b0def2106d95532369bc4df7df3c6cffdcad9e98c20be9d55a3bf680b2c97"} Jun 03 15:57:10.413692 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.413624 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" event={"ID":"490f0a30-89ae-4e18-87f0-498a714ff08c","Type":"ContainerStarted","Data":"10194eb1dfaaf00cb88047cb840d68a1195ce7dd5d1d1cec4788eea8e489a18a"} Jun 03 15:57:10.426294 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.426226 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z289q" event={"ID":"c1c44950-8787-4b29-9f1b-c6a191f81431","Type":"ContainerStarted","Data":"ed5803fc00aca3aee112b65f84134e52f4a02e83dbfa931b3b562ff11393c50e"} Jun 03 15:57:10.435271 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.435189 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tdzqn" event={"ID":"31941200-bc35-48cc-9148-b104dbfde42e","Type":"ContainerStarted","Data":"5fc41f5f30f03b4c29e2a7139b1719cb89920766c9482f5dfd6b91ae922a6a1d"} Jun 03 15:57:10.437526 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.437464 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4ppxs" event={"ID":"0aa350a0-203c-4286-b184-d62efbd0af97","Type":"ContainerStarted","Data":"feb3302c3386fc149156d8244cb3af196d65c40b498c6b8dad4d505617e359dd"} Jun 03 15:57:10.451013 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.450376 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-61.ec2.internal" event={"ID":"790a10cb9330d251bd135e9345ddb3cd","Type":"ContainerStarted","Data":"01e71a3d3a75210815466e91e86c388818227ba834baae324a4a0a9f99c075ae"} Jun 03 15:57:10.466826 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.466578 2576 generic.go:358] "Generic (PLEG): container finished" podID="cdf3ba6c66aaec9ab5812e4869ec3138" containerID="8a9b06201171b8ecadd09fc1f0e0e6c335d1131b75a12d5f6244f3291e400e02" exitCode=0 Jun 03 15:57:10.466826 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.466623 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal" event={"ID":"cdf3ba6c66aaec9ab5812e4869ec3138","Type":"ContainerDied","Data":"8a9b06201171b8ecadd09fc1f0e0e6c335d1131b75a12d5f6244f3291e400e02"} Jun 03 15:57:10.479889 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.478898 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-61.ec2.internal" podStartSLOduration=2.478879928 podStartE2EDuration="2.478879928s" podCreationTimestamp="2026-06-03 15:57:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 15:57:10.464762012 +0000 UTC m=+3.641731838" watchObservedRunningTime="2026-06-03 15:57:10.478879928 +0000 UTC m=+3.655849740" Jun 03 15:57:10.875949 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.875909 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs\") pod \"network-metrics-daemon-w5jsj\" (UID: \"2b455f74-b695-4158-8fe3-2fe335bd3e3b\") " pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:10.876245 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:10.876072 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:57:10.876245 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:10.876152 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs podName:2b455f74-b695-4158-8fe3-2fe335bd3e3b nodeName:}" failed. No retries permitted until 2026-06-03 15:57:12.876130978 +0000 UTC m=+6.053100779 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs") pod "network-metrics-daemon-w5jsj" (UID: "2b455f74-b695-4158-8fe3-2fe335bd3e3b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:57:10.976552 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:10.976515 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hn4zm\" (UniqueName: \"kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm\") pod \"network-check-target-g74tp\" (UID: \"5bcd65d8-45ad-42e2-8554-16e218c5b465\") " pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:10.976761 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:10.976744 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 15:57:10.976860 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:10.976783 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 15:57:10.976860 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:10.976796 2576 projected.go:194] Error preparing data for projected volume kube-api-access-hn4zm for pod openshift-network-diagnostics/network-check-target-g74tp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:57:10.976860 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:10.976856 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm podName:5bcd65d8-45ad-42e2-8554-16e218c5b465 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:12.976837541 +0000 UTC m=+6.153807334 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hn4zm" (UniqueName: "kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm") pod "network-check-target-g74tp" (UID: "5bcd65d8-45ad-42e2-8554-16e218c5b465") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:57:11.390463 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:11.390431 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:11.391204 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:11.391167 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:11.391877 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:11.391857 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:11.392154 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:11.392134 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:11.492869 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:11.492813 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal" event={"ID":"cdf3ba6c66aaec9ab5812e4869ec3138","Type":"ContainerStarted","Data":"5208aebb96f4009b4bc6a10452635b1320f0aad5c93ff0878a0b13de62ec7784"} Jun 03 15:57:11.506552 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:11.506491 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-61.ec2.internal" podStartSLOduration=3.5064727319999998 podStartE2EDuration="3.506472732s" podCreationTimestamp="2026-06-03 15:57:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 15:57:11.505986735 +0000 UTC m=+4.682956545" watchObservedRunningTime="2026-06-03 15:57:11.506472732 +0000 UTC m=+4.683442543" Jun 03 15:57:12.892287 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:12.891706 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs\") pod \"network-metrics-daemon-w5jsj\" (UID: \"2b455f74-b695-4158-8fe3-2fe335bd3e3b\") " pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:12.892287 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:12.891878 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:57:12.892287 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:12.891946 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs podName:2b455f74-b695-4158-8fe3-2fe335bd3e3b nodeName:}" failed. No retries permitted until 2026-06-03 15:57:16.891926533 +0000 UTC m=+10.068896335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs") pod "network-metrics-daemon-w5jsj" (UID: "2b455f74-b695-4158-8fe3-2fe335bd3e3b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:57:12.993166 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:12.992927 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hn4zm\" (UniqueName: \"kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm\") pod \"network-check-target-g74tp\" (UID: \"5bcd65d8-45ad-42e2-8554-16e218c5b465\") " pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:12.993166 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:12.993045 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 15:57:12.993166 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:12.993059 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 15:57:12.993166 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:12.993068 2576 projected.go:194] Error preparing data for projected volume kube-api-access-hn4zm for pod openshift-network-diagnostics/network-check-target-g74tp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:57:12.993166 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:12.993118 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm podName:5bcd65d8-45ad-42e2-8554-16e218c5b465 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:16.993100603 +0000 UTC m=+10.170070410 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hn4zm" (UniqueName: "kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm") pod "network-check-target-g74tp" (UID: "5bcd65d8-45ad-42e2-8554-16e218c5b465") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:57:13.385900 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:13.385865 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:13.386094 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:13.386000 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:13.386592 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:13.386570 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:13.386697 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:13.386682 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:15.390280 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:15.389638 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:15.390280 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:15.389788 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:15.390280 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:15.390153 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:15.390280 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:15.390238 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:16.929223 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:16.929148 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs\") pod \"network-metrics-daemon-w5jsj\" (UID: \"2b455f74-b695-4158-8fe3-2fe335bd3e3b\") " pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:16.929683 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:16.929284 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:57:16.929683 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:16.929345 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs podName:2b455f74-b695-4158-8fe3-2fe335bd3e3b nodeName:}" failed. No retries permitted until 2026-06-03 15:57:24.929326845 +0000 UTC m=+18.106296632 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs") pod "network-metrics-daemon-w5jsj" (UID: "2b455f74-b695-4158-8fe3-2fe335bd3e3b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:57:17.029979 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:17.029906 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hn4zm\" (UniqueName: \"kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm\") pod \"network-check-target-g74tp\" (UID: \"5bcd65d8-45ad-42e2-8554-16e218c5b465\") " pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:17.030145 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:17.030103 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 15:57:17.030145 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:17.030122 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 15:57:17.030145 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:17.030134 2576 projected.go:194] Error preparing data for projected volume kube-api-access-hn4zm for pod openshift-network-diagnostics/network-check-target-g74tp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:57:17.030338 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:17.030191 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm podName:5bcd65d8-45ad-42e2-8554-16e218c5b465 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:25.030172775 +0000 UTC m=+18.207142580 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hn4zm" (UniqueName: "kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm") pod "network-check-target-g74tp" (UID: "5bcd65d8-45ad-42e2-8554-16e218c5b465") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:57:17.386458 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:17.386423 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:17.386687 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:17.386548 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:17.387046 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:17.387026 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:17.387161 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:17.387140 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:19.335952 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:19.335917 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-zfdzl"] Jun 03 15:57:19.344964 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:19.344935 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:19.345126 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:19.345006 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfdzl" podUID="d8f8466e-baef-4149-964b-a5366d798c52" Jun 03 15:57:19.387888 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:19.387857 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:19.388036 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:19.387857 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:19.388036 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:19.387980 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:19.388150 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:19.388076 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:19.449339 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:19.449303 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d8f8466e-baef-4149-964b-a5366d798c52-dbus\") pod \"global-pull-secret-syncer-zfdzl\" (UID: \"d8f8466e-baef-4149-964b-a5366d798c52\") " pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:19.449535 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:19.449353 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret\") pod \"global-pull-secret-syncer-zfdzl\" (UID: \"d8f8466e-baef-4149-964b-a5366d798c52\") " pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:19.449535 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:19.449399 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d8f8466e-baef-4149-964b-a5366d798c52-kubelet-config\") pod \"global-pull-secret-syncer-zfdzl\" (UID: \"d8f8466e-baef-4149-964b-a5366d798c52\") " pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:19.549931 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:19.549894 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d8f8466e-baef-4149-964b-a5366d798c52-dbus\") pod \"global-pull-secret-syncer-zfdzl\" (UID: \"d8f8466e-baef-4149-964b-a5366d798c52\") " pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:19.549931 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:19.549929 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret\") pod \"global-pull-secret-syncer-zfdzl\" (UID: \"d8f8466e-baef-4149-964b-a5366d798c52\") " pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:19.550170 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:19.549961 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d8f8466e-baef-4149-964b-a5366d798c52-kubelet-config\") pod \"global-pull-secret-syncer-zfdzl\" (UID: \"d8f8466e-baef-4149-964b-a5366d798c52\") " pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:19.550170 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:19.550038 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d8f8466e-baef-4149-964b-a5366d798c52-kubelet-config\") pod \"global-pull-secret-syncer-zfdzl\" (UID: \"d8f8466e-baef-4149-964b-a5366d798c52\") " pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:19.550170 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:19.550058 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 15:57:19.550170 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:19.550120 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret podName:d8f8466e-baef-4149-964b-a5366d798c52 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:20.05010205 +0000 UTC m=+13.227071858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret") pod "global-pull-secret-syncer-zfdzl" (UID: "d8f8466e-baef-4149-964b-a5366d798c52") : object "kube-system"/"original-pull-secret" not registered Jun 03 15:57:19.550170 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:19.550134 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d8f8466e-baef-4149-964b-a5366d798c52-dbus\") pod \"global-pull-secret-syncer-zfdzl\" (UID: \"d8f8466e-baef-4149-964b-a5366d798c52\") " pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:20.053136 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:20.053103 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret\") pod \"global-pull-secret-syncer-zfdzl\" (UID: \"d8f8466e-baef-4149-964b-a5366d798c52\") " pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:20.053307 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:20.053227 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 15:57:20.053307 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:20.053282 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret podName:d8f8466e-baef-4149-964b-a5366d798c52 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:21.053268372 +0000 UTC m=+14.230238159 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret") pod "global-pull-secret-syncer-zfdzl" (UID: "d8f8466e-baef-4149-964b-a5366d798c52") : object "kube-system"/"original-pull-secret" not registered Jun 03 15:57:20.385194 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:20.385163 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:20.385606 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:20.385295 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfdzl" podUID="d8f8466e-baef-4149-964b-a5366d798c52" Jun 03 15:57:21.059073 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:21.059038 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret\") pod \"global-pull-secret-syncer-zfdzl\" (UID: \"d8f8466e-baef-4149-964b-a5366d798c52\") " pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:21.059273 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:21.059195 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 15:57:21.059326 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:21.059279 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret podName:d8f8466e-baef-4149-964b-a5366d798c52 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:23.059259431 +0000 UTC m=+16.236229242 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret") pod "global-pull-secret-syncer-zfdzl" (UID: "d8f8466e-baef-4149-964b-a5366d798c52") : object "kube-system"/"original-pull-secret" not registered Jun 03 15:57:21.387644 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:21.387567 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:21.388040 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:21.387684 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:21.388040 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:21.387738 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:21.388040 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:21.387839 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:22.385208 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:22.385170 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:22.385386 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:22.385288 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfdzl" podUID="d8f8466e-baef-4149-964b-a5366d798c52" Jun 03 15:57:23.075047 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:23.075006 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret\") pod \"global-pull-secret-syncer-zfdzl\" (UID: \"d8f8466e-baef-4149-964b-a5366d798c52\") " pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:23.075476 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:23.075178 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 15:57:23.075476 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:23.075260 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret podName:d8f8466e-baef-4149-964b-a5366d798c52 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:27.075238513 +0000 UTC m=+20.252208307 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret") pod "global-pull-secret-syncer-zfdzl" (UID: "d8f8466e-baef-4149-964b-a5366d798c52") : object "kube-system"/"original-pull-secret" not registered Jun 03 15:57:23.385066 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:23.384974 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:23.385066 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:23.385010 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:23.385281 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:23.385113 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:23.385281 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:23.385249 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:24.384788 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:24.384744 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:24.385177 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:24.384894 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfdzl" podUID="d8f8466e-baef-4149-964b-a5366d798c52" Jun 03 15:57:24.986962 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:24.986927 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs\") pod \"network-metrics-daemon-w5jsj\" (UID: \"2b455f74-b695-4158-8fe3-2fe335bd3e3b\") " pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:24.987138 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:24.987094 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:57:24.987189 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:24.987158 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs podName:2b455f74-b695-4158-8fe3-2fe335bd3e3b nodeName:}" failed. No retries permitted until 2026-06-03 15:57:40.987139547 +0000 UTC m=+34.164109341 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs") pod "network-metrics-daemon-w5jsj" (UID: "2b455f74-b695-4158-8fe3-2fe335bd3e3b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:57:25.087728 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:25.087688 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hn4zm\" (UniqueName: \"kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm\") pod \"network-check-target-g74tp\" (UID: \"5bcd65d8-45ad-42e2-8554-16e218c5b465\") " pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:25.087894 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:25.087879 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 15:57:25.087942 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:25.087900 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 15:57:25.087942 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:25.087911 2576 projected.go:194] Error preparing data for projected volume kube-api-access-hn4zm for pod openshift-network-diagnostics/network-check-target-g74tp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:57:25.088047 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:25.087976 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm podName:5bcd65d8-45ad-42e2-8554-16e218c5b465 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:41.087957068 +0000 UTC m=+34.264926861 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hn4zm" (UniqueName: "kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm") pod "network-check-target-g74tp" (UID: "5bcd65d8-45ad-42e2-8554-16e218c5b465") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:57:25.385021 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:25.384923 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:25.385021 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:25.384975 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:25.385504 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:25.385052 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:25.385504 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:25.385483 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:26.385348 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:26.385304 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:26.385815 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:26.385445 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfdzl" podUID="d8f8466e-baef-4149-964b-a5366d798c52" Jun 03 15:57:27.103802 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:27.102728 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret\") pod \"global-pull-secret-syncer-zfdzl\" (UID: \"d8f8466e-baef-4149-964b-a5366d798c52\") " pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:27.103802 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:27.103519 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 15:57:27.103802 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:27.103611 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret podName:d8f8466e-baef-4149-964b-a5366d798c52 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:35.103590597 +0000 UTC m=+28.280560388 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret") pod "global-pull-secret-syncer-zfdzl" (UID: "d8f8466e-baef-4149-964b-a5366d798c52") : object "kube-system"/"original-pull-secret" not registered Jun 03 15:57:27.387577 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:27.387508 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:27.387577 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:27.387527 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:27.388003 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:27.387609 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:27.388003 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:27.387745 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:28.385070 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.384860 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:28.385213 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:28.385157 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfdzl" podUID="d8f8466e-baef-4149-964b-a5366d798c52" Jun 03 15:57:28.525317 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.525075 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" event={"ID":"490f0a30-89ae-4e18-87f0-498a714ff08c","Type":"ContainerStarted","Data":"68787ed4dd13bc0a0e615fea44788898ec51d43c74d45e8583656a43efda8753"} Jun 03 15:57:28.526099 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.525333 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" event={"ID":"490f0a30-89ae-4e18-87f0-498a714ff08c","Type":"ContainerStarted","Data":"c30e973fba896bef899e91999ec98d557bb804d45f25b940f8734fe9136d82de"} Jun 03 15:57:28.526099 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.525353 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" event={"ID":"490f0a30-89ae-4e18-87f0-498a714ff08c","Type":"ContainerStarted","Data":"c17cb54e82cb52e3060373e3675bb216029cdff78fb35c694737398ad95e8aad"} Jun 03 15:57:28.526099 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.525365 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" event={"ID":"490f0a30-89ae-4e18-87f0-498a714ff08c","Type":"ContainerStarted","Data":"a66d583ccbdd4f17d10b4f9519c159c53e3d571bfa7b84b2c4cbdb6e3c785d0f"} Jun 03 15:57:28.526431 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.526411 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tdzqn" event={"ID":"31941200-bc35-48cc-9148-b104dbfde42e","Type":"ContainerStarted","Data":"dbd07d5170480a3365754ebf95f96105733a8a074fef240d5bbfbc1d7270df24"} Jun 03 15:57:28.527858 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.527835 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4ppxs" event={"ID":"0aa350a0-203c-4286-b184-d62efbd0af97","Type":"ContainerStarted","Data":"5c055fce3bbe0efc596446224a8f446f6149920681362605d1313d68260d6dea"} Jun 03 15:57:28.529116 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.529091 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" event={"ID":"2544ae4d-5cee-47ff-b8db-2e27aefebf5c","Type":"ContainerStarted","Data":"0ac0861af7ec7d66ea3695de6d3a3673976ceab48e114630e3249dae8c83c265"} Jun 03 15:57:28.530576 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.530550 2576 generic.go:358] "Generic (PLEG): container finished" podID="27386c54-e43c-47d4-89e2-7ae61fb82e65" containerID="0007dcc1142c8a90a88097a6e62eca737dcdb62593177cbe4b46bd2d36ae532e" exitCode=0 Jun 03 15:57:28.530669 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.530593 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7xtj" event={"ID":"27386c54-e43c-47d4-89e2-7ae61fb82e65","Type":"ContainerDied","Data":"0007dcc1142c8a90a88097a6e62eca737dcdb62593177cbe4b46bd2d36ae532e"} Jun 03 15:57:28.532215 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.531940 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" event={"ID":"7e1d606f-ce5b-4871-b678-8d93934e06bf","Type":"ContainerStarted","Data":"d61cd9f595d9c40ec4541a322420c1e84c8bf3690fbf96461527d2477800dc81"} Jun 03 15:57:28.533311 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.533252 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q5cmz" event={"ID":"16c438af-88c9-4b06-be30-c984a697afeb","Type":"ContainerStarted","Data":"faac22baecb7e4176d1cf863da0d5d38c07e99c78a39de6d32d258c9127ad25c"} Jun 03 15:57:28.534540 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.534519 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7w5ch" event={"ID":"128d979f-9fbc-4899-8f0a-b0023e619799","Type":"ContainerStarted","Data":"da8568a121c5dbc40c335a4dd078c7cd8427293738469903fb0070b894588517"} Jun 03 15:57:28.546497 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.546450 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-tdzqn" podStartSLOduration=3.507063844 podStartE2EDuration="21.54643365s" podCreationTimestamp="2026-06-03 15:57:07 +0000 UTC" firstStartedPulling="2026-06-03 15:57:09.98573943 +0000 UTC m=+3.162709220" lastFinishedPulling="2026-06-03 15:57:28.025109234 +0000 UTC m=+21.202079026" observedRunningTime="2026-06-03 15:57:28.546182686 +0000 UTC m=+21.723152494" watchObservedRunningTime="2026-06-03 15:57:28.54643365 +0000 UTC m=+21.723403463" Jun 03 15:57:28.560803 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.560736 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4ppxs" podStartSLOduration=3.533762188 podStartE2EDuration="21.560721273s" podCreationTimestamp="2026-06-03 15:57:07 +0000 UTC" firstStartedPulling="2026-06-03 15:57:09.977864587 +0000 UTC m=+3.154834376" lastFinishedPulling="2026-06-03 15:57:28.004823659 +0000 UTC m=+21.181793461" observedRunningTime="2026-06-03 15:57:28.560348742 +0000 UTC m=+21.737318552" watchObservedRunningTime="2026-06-03 15:57:28.560721273 +0000 UTC m=+21.737691082" Jun 03 15:57:28.592756 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.592713 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-p4fdl" podStartSLOduration=3.5682097759999998 podStartE2EDuration="21.592699332s" podCreationTimestamp="2026-06-03 15:57:07 +0000 UTC" firstStartedPulling="2026-06-03 15:57:09.982247368 +0000 UTC m=+3.159217159" lastFinishedPulling="2026-06-03 15:57:28.006736925 +0000 UTC m=+21.183706715" observedRunningTime="2026-06-03 15:57:28.592439471 +0000 UTC m=+21.769409282" watchObservedRunningTime="2026-06-03 15:57:28.592699332 +0000 UTC m=+21.769669141" Jun 03 15:57:28.593002 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.592826 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-q5cmz" podStartSLOduration=3.567684987 podStartE2EDuration="21.592821237s" podCreationTimestamp="2026-06-03 15:57:07 +0000 UTC" firstStartedPulling="2026-06-03 15:57:09.980125207 +0000 UTC m=+3.157094994" lastFinishedPulling="2026-06-03 15:57:28.005261452 +0000 UTC m=+21.182231244" observedRunningTime="2026-06-03 15:57:28.576277772 +0000 UTC m=+21.753247580" watchObservedRunningTime="2026-06-03 15:57:28.592821237 +0000 UTC m=+21.769791048" Jun 03 15:57:28.626486 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:28.626430 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7w5ch" podStartSLOduration=11.706562935000001 podStartE2EDuration="21.626414177s" podCreationTimestamp="2026-06-03 15:57:07 +0000 UTC" firstStartedPulling="2026-06-03 15:57:09.97822559 +0000 UTC m=+3.155195383" lastFinishedPulling="2026-06-03 15:57:19.898076825 +0000 UTC m=+13.075046625" observedRunningTime="2026-06-03 15:57:28.625883632 +0000 UTC m=+21.802853464" watchObservedRunningTime="2026-06-03 15:57:28.626414177 +0000 UTC m=+21.803383985" Jun 03 15:57:29.155303 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:29.155275 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 03 15:57:29.321281 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:29.321118 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-03T15:57:29.155297919Z","UUID":"5a47356a-0fad-4202-9c3a-518121da1835","Handler":null,"Name":"","Endpoint":""} Jun 03 15:57:29.323845 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:29.323821 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 03 15:57:29.323969 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:29.323854 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 03 15:57:29.385511 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:29.385447 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:29.385721 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:29.385578 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:29.385830 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:29.385455 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:29.385945 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:29.385924 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:29.539396 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:29.539363 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" event={"ID":"2544ae4d-5cee-47ff-b8db-2e27aefebf5c","Type":"ContainerStarted","Data":"f4166e29322ba0cc01c664ca6acfbaa1f8a48ff0c86880dd1a72914fabddbdad"} Jun 03 15:57:29.544663 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:29.544591 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" event={"ID":"490f0a30-89ae-4e18-87f0-498a714ff08c","Type":"ContainerStarted","Data":"21830c4999c66589535a6e7896675033e34fe03875f24e4af23de1f3ecdfba8c"} Jun 03 15:57:29.544663 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:29.544631 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" event={"ID":"490f0a30-89ae-4e18-87f0-498a714ff08c","Type":"ContainerStarted","Data":"050c5aaff34f24d92f9c196679477661849592059cb6d1a7aa18fc6efb48d2a1"} Jun 03 15:57:29.562036 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:29.561956 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z289q" podStartSLOduration=4.54259643 podStartE2EDuration="22.561939862s" podCreationTimestamp="2026-06-03 15:57:07 +0000 UTC" firstStartedPulling="2026-06-03 15:57:09.985904401 +0000 UTC m=+3.162874187" lastFinishedPulling="2026-06-03 15:57:28.005247833 +0000 UTC m=+21.182217619" observedRunningTime="2026-06-03 15:57:29.561725428 +0000 UTC m=+22.738695237" watchObservedRunningTime="2026-06-03 15:57:29.561939862 +0000 UTC m=+22.738909675" Jun 03 15:57:30.384992 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:30.384953 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:30.385189 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:30.385080 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfdzl" podUID="d8f8466e-baef-4149-964b-a5366d798c52" Jun 03 15:57:30.551013 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:30.550972 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" event={"ID":"2544ae4d-5cee-47ff-b8db-2e27aefebf5c","Type":"ContainerStarted","Data":"ff22886cdd3b8caf5c38a425bb7c88ca80384796c4c05f800087922efbf8189e"} Jun 03 15:57:30.552342 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:30.552316 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z289q" event={"ID":"c1c44950-8787-4b29-9f1b-c6a191f81431","Type":"ContainerStarted","Data":"28b07b7f1669a9a1f8940465b3d926c19363058db112b3954196055a17bb95b9"} Jun 03 15:57:30.567460 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:30.567410 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dm5xr" podStartSLOduration=3.6337766670000002 podStartE2EDuration="23.567395311s" podCreationTimestamp="2026-06-03 15:57:07 +0000 UTC" firstStartedPulling="2026-06-03 15:57:09.98419465 +0000 UTC m=+3.161164451" lastFinishedPulling="2026-06-03 15:57:29.917813305 +0000 UTC m=+23.094783095" observedRunningTime="2026-06-03 15:57:30.567315862 +0000 UTC m=+23.744285672" watchObservedRunningTime="2026-06-03 15:57:30.567395311 +0000 UTC m=+23.744365120" Jun 03 15:57:30.906401 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:30.906362 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7w5ch" Jun 03 15:57:30.907079 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:30.907015 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7w5ch" Jun 03 15:57:31.388041 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:31.388011 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:31.388223 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:31.388011 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:31.388223 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:31.388130 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:31.388337 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:31.388233 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:31.558084 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:31.558047 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" event={"ID":"490f0a30-89ae-4e18-87f0-498a714ff08c","Type":"ContainerStarted","Data":"d3b14cc206a0372e43fe5cae7ef6f25c12d0c231fb8e0c0ba7da78f56e8a65fa"} Jun 03 15:57:32.384605 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:32.384568 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:32.384795 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:32.384678 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfdzl" podUID="d8f8466e-baef-4149-964b-a5366d798c52" Jun 03 15:57:33.388034 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:33.387862 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:33.388713 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:33.388101 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:33.388713 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:33.387862 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:33.388713 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:33.388169 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:33.563468 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:33.563436 2576 generic.go:358] "Generic (PLEG): container finished" podID="27386c54-e43c-47d4-89e2-7ae61fb82e65" containerID="b05382b78bdaf1ce64efe6e30c62ce92a49c8d12330afcb24aec976a2de18aa1" exitCode=0 Jun 03 15:57:33.563646 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:33.563530 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7xtj" event={"ID":"27386c54-e43c-47d4-89e2-7ae61fb82e65","Type":"ContainerDied","Data":"b05382b78bdaf1ce64efe6e30c62ce92a49c8d12330afcb24aec976a2de18aa1"} Jun 03 15:57:33.566960 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:33.566935 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" event={"ID":"490f0a30-89ae-4e18-87f0-498a714ff08c","Type":"ContainerStarted","Data":"f84445af15ea16646868abe30cdcb06a7876babe395247015e296a95b0f2797b"} Jun 03 15:57:33.567310 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:33.567293 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:33.582739 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:33.582715 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:33.608373 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:33.608325 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" podStartSLOduration=8.525998818 podStartE2EDuration="26.608310483s" podCreationTimestamp="2026-06-03 15:57:07 +0000 UTC" firstStartedPulling="2026-06-03 15:57:09.975154699 +0000 UTC m=+3.152124486" lastFinishedPulling="2026-06-03 15:57:28.057466353 +0000 UTC m=+21.234436151" observedRunningTime="2026-06-03 15:57:33.607543514 +0000 UTC m=+26.784513320" watchObservedRunningTime="2026-06-03 15:57:33.608310483 +0000 UTC m=+26.785280292" Jun 03 15:57:34.385155 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:34.385120 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:34.385311 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:34.385224 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfdzl" podUID="d8f8466e-baef-4149-964b-a5366d798c52" Jun 03 15:57:34.568963 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:34.568933 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:34.568963 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:34.568968 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:34.584132 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:34.584108 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:57:34.755741 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:34.755266 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7w5ch" Jun 03 15:57:34.755949 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:34.755910 2576 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 03 15:57:34.756476 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:34.756236 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7w5ch" Jun 03 15:57:34.764751 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:34.764724 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g74tp"] Jun 03 15:57:34.764894 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:34.764858 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:34.764973 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:34.764947 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:34.767589 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:34.767566 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w5jsj"] Jun 03 15:57:34.767696 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:34.767685 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:34.767881 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:34.767840 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:34.768247 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:34.768229 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zfdzl"] Jun 03 15:57:34.768360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:34.768331 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:34.768446 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:34.768423 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfdzl" podUID="d8f8466e-baef-4149-964b-a5366d798c52" Jun 03 15:57:35.160819 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:35.160754 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret\") pod \"global-pull-secret-syncer-zfdzl\" (UID: \"d8f8466e-baef-4149-964b-a5366d798c52\") " pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:35.160986 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:35.160885 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 15:57:35.160986 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:35.160951 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret podName:d8f8466e-baef-4149-964b-a5366d798c52 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:51.160934455 +0000 UTC m=+44.337904249 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret") pod "global-pull-secret-syncer-zfdzl" (UID: "d8f8466e-baef-4149-964b-a5366d798c52") : object "kube-system"/"original-pull-secret" not registered Jun 03 15:57:35.571901 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:35.571867 2576 generic.go:358] "Generic (PLEG): container finished" podID="27386c54-e43c-47d4-89e2-7ae61fb82e65" containerID="2d2a300ce79e1a69dd95a453786d133826e89ef4bc16efb4309aeed7771cdd91" exitCode=0 Jun 03 15:57:35.572321 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:35.571902 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7xtj" event={"ID":"27386c54-e43c-47d4-89e2-7ae61fb82e65","Type":"ContainerDied","Data":"2d2a300ce79e1a69dd95a453786d133826e89ef4bc16efb4309aeed7771cdd91"} Jun 03 15:57:36.384477 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:36.384443 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:36.384654 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:36.384572 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfdzl" podUID="d8f8466e-baef-4149-964b-a5366d798c52" Jun 03 15:57:36.384868 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:36.384841 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:36.384993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:36.384845 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:36.385059 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:36.385040 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:36.385111 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:36.385052 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:37.578033 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:37.577999 2576 generic.go:358] "Generic (PLEG): container finished" podID="27386c54-e43c-47d4-89e2-7ae61fb82e65" containerID="401cdc9588f5fdd9881c5ada1c6eb66efce2cb18b110f7fea478b0a060b0cd85" exitCode=0 Jun 03 15:57:37.578530 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:37.578062 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7xtj" event={"ID":"27386c54-e43c-47d4-89e2-7ae61fb82e65","Type":"ContainerDied","Data":"401cdc9588f5fdd9881c5ada1c6eb66efce2cb18b110f7fea478b0a060b0cd85"} Jun 03 15:57:38.384620 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:38.384583 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:38.384620 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:38.384603 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:38.384843 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:38.384694 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfdzl" podUID="d8f8466e-baef-4149-964b-a5366d798c52" Jun 03 15:57:38.384843 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:38.384742 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:38.384937 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:38.384856 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:38.385009 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:38.384985 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:40.384690 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.384659 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:40.385274 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.384659 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:40.385274 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:40.384787 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfdzl" podUID="d8f8466e-baef-4149-964b-a5366d798c52" Jun 03 15:57:40.385274 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.384800 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:40.385274 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:40.384890 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g74tp" podUID="5bcd65d8-45ad-42e2-8554-16e218c5b465" Jun 03 15:57:40.385274 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:40.384957 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5jsj" podUID="2b455f74-b695-4158-8fe3-2fe335bd3e3b" Jun 03 15:57:40.688142 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.688111 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-61.ec2.internal" event="NodeReady" Jun 03 15:57:40.688303 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.688268 2576 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 03 15:57:40.731710 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.731673 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-76kk6"] Jun 03 15:57:40.747458 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.747427 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-l2jxd"] Jun 03 15:57:40.747633 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.747597 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:40.750155 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.750126 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vbxw8\"" Jun 03 15:57:40.750312 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.750164 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 03 15:57:40.750312 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.750301 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 03 15:57:40.773308 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.773274 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-76kk6"] Jun 03 15:57:40.773308 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.773312 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l2jxd"] Jun 03 15:57:40.773510 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.773415 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:57:40.775925 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.775897 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 03 15:57:40.776047 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.775897 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 03 15:57:40.776047 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.776005 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 03 15:57:40.776240 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.776224 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-drwlt\"" Jun 03 15:57:40.802407 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.802374 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:40.802574 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.802419 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gpbt\" (UniqueName: \"kubernetes.io/projected/649647f7-81ce-4ed1-80d4-7ab2cabda43d-kube-api-access-4gpbt\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:40.802574 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.802522 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/649647f7-81ce-4ed1-80d4-7ab2cabda43d-config-volume\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:40.802672 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.802591 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/649647f7-81ce-4ed1-80d4-7ab2cabda43d-tmp-dir\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:40.903739 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.903697 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/649647f7-81ce-4ed1-80d4-7ab2cabda43d-tmp-dir\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:40.903739 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.903745 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:40.903993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.903815 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4gpbt\" (UniqueName: \"kubernetes.io/projected/649647f7-81ce-4ed1-80d4-7ab2cabda43d-kube-api-access-4gpbt\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:40.903993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.903854 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw5lg\" (UniqueName: \"kubernetes.io/projected/1e3e630c-6304-40da-935c-2dc17eb2ddd0-kube-api-access-zw5lg\") pod \"ingress-canary-l2jxd\" (UID: \"1e3e630c-6304-40da-935c-2dc17eb2ddd0\") " pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:57:40.903993 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:40.903869 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 15:57:40.903993 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.903890 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert\") pod \"ingress-canary-l2jxd\" (UID: \"1e3e630c-6304-40da-935c-2dc17eb2ddd0\") " pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:57:40.903993 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:40.903928 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls podName:649647f7-81ce-4ed1-80d4-7ab2cabda43d nodeName:}" failed. No retries permitted until 2026-06-03 15:57:41.403909026 +0000 UTC m=+34.580878834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls") pod "dns-default-76kk6" (UID: "649647f7-81ce-4ed1-80d4-7ab2cabda43d") : secret "dns-default-metrics-tls" not found Jun 03 15:57:40.904218 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.903993 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/649647f7-81ce-4ed1-80d4-7ab2cabda43d-config-volume\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:40.904516 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.904492 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/649647f7-81ce-4ed1-80d4-7ab2cabda43d-tmp-dir\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:40.904727 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.904706 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/649647f7-81ce-4ed1-80d4-7ab2cabda43d-config-volume\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:40.915595 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:40.915461 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gpbt\" (UniqueName: \"kubernetes.io/projected/649647f7-81ce-4ed1-80d4-7ab2cabda43d-kube-api-access-4gpbt\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:41.004658 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:41.004568 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs\") pod \"network-metrics-daemon-w5jsj\" (UID: \"2b455f74-b695-4158-8fe3-2fe335bd3e3b\") " pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:41.004658 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:41.004620 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zw5lg\" (UniqueName: \"kubernetes.io/projected/1e3e630c-6304-40da-935c-2dc17eb2ddd0-kube-api-access-zw5lg\") pod \"ingress-canary-l2jxd\" (UID: \"1e3e630c-6304-40da-935c-2dc17eb2ddd0\") " pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:57:41.004658 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:41.004645 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert\") pod \"ingress-canary-l2jxd\" (UID: \"1e3e630c-6304-40da-935c-2dc17eb2ddd0\") " pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:57:41.004944 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:41.004745 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 15:57:41.004944 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:41.004740 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:57:41.004944 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:41.004827 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert podName:1e3e630c-6304-40da-935c-2dc17eb2ddd0 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:41.504809051 +0000 UTC m=+34.681778852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert") pod "ingress-canary-l2jxd" (UID: "1e3e630c-6304-40da-935c-2dc17eb2ddd0") : secret "canary-serving-cert" not found Jun 03 15:57:41.004944 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:41.004844 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs podName:2b455f74-b695-4158-8fe3-2fe335bd3e3b nodeName:}" failed. No retries permitted until 2026-06-03 15:58:13.004836057 +0000 UTC m=+66.181805848 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs") pod "network-metrics-daemon-w5jsj" (UID: "2b455f74-b695-4158-8fe3-2fe335bd3e3b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 15:57:41.015392 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:41.015363 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw5lg\" (UniqueName: \"kubernetes.io/projected/1e3e630c-6304-40da-935c-2dc17eb2ddd0-kube-api-access-zw5lg\") pod \"ingress-canary-l2jxd\" (UID: \"1e3e630c-6304-40da-935c-2dc17eb2ddd0\") " pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:57:41.105808 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:41.105756 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hn4zm\" (UniqueName: \"kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm\") pod \"network-check-target-g74tp\" (UID: \"5bcd65d8-45ad-42e2-8554-16e218c5b465\") " pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:41.105997 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:41.105916 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 15:57:41.105997 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:41.105942 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 15:57:41.105997 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:41.105954 2576 projected.go:194] Error preparing data for projected volume kube-api-access-hn4zm for pod openshift-network-diagnostics/network-check-target-g74tp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:57:41.106155 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:41.106019 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm podName:5bcd65d8-45ad-42e2-8554-16e218c5b465 nodeName:}" failed. No retries permitted until 2026-06-03 15:58:13.106000375 +0000 UTC m=+66.282970164 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hn4zm" (UniqueName: "kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm") pod "network-check-target-g74tp" (UID: "5bcd65d8-45ad-42e2-8554-16e218c5b465") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 15:57:41.408279 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:41.408196 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:41.408671 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:41.408334 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 15:57:41.408671 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:41.408402 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls podName:649647f7-81ce-4ed1-80d4-7ab2cabda43d nodeName:}" failed. No retries permitted until 2026-06-03 15:57:42.408385985 +0000 UTC m=+35.585355772 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls") pod "dns-default-76kk6" (UID: "649647f7-81ce-4ed1-80d4-7ab2cabda43d") : secret "dns-default-metrics-tls" not found Jun 03 15:57:41.509188 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:41.509150 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert\") pod \"ingress-canary-l2jxd\" (UID: \"1e3e630c-6304-40da-935c-2dc17eb2ddd0\") " pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:57:41.509371 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:41.509326 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 15:57:41.509427 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:41.509384 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert podName:1e3e630c-6304-40da-935c-2dc17eb2ddd0 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:42.509369102 +0000 UTC m=+35.686338889 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert") pod "ingress-canary-l2jxd" (UID: "1e3e630c-6304-40da-935c-2dc17eb2ddd0") : secret "canary-serving-cert" not found Jun 03 15:57:42.385286 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:42.385246 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:42.385476 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:42.385246 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:57:42.385553 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:42.385245 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:57:42.389233 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:42.389203 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 15:57:42.389366 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:42.389269 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 15:57:42.389366 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:42.389307 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 03 15:57:42.389366 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:42.389205 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 15:57:42.389937 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:42.389851 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qk87p\"" Jun 03 15:57:42.390036 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:42.390016 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xx6f7\"" Jun 03 15:57:42.418392 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:42.418357 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:42.418786 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:42.418531 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 15:57:42.418786 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:42.418609 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls podName:649647f7-81ce-4ed1-80d4-7ab2cabda43d nodeName:}" failed. No retries permitted until 2026-06-03 15:57:44.41858683 +0000 UTC m=+37.595556621 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls") pod "dns-default-76kk6" (UID: "649647f7-81ce-4ed1-80d4-7ab2cabda43d") : secret "dns-default-metrics-tls" not found Jun 03 15:57:42.519185 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:42.519142 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert\") pod \"ingress-canary-l2jxd\" (UID: \"1e3e630c-6304-40da-935c-2dc17eb2ddd0\") " pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:57:42.519371 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:42.519298 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 15:57:42.519371 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:42.519370 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert podName:1e3e630c-6304-40da-935c-2dc17eb2ddd0 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:44.51935094 +0000 UTC m=+37.696320746 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert") pod "ingress-canary-l2jxd" (UID: "1e3e630c-6304-40da-935c-2dc17eb2ddd0") : secret "canary-serving-cert" not found Jun 03 15:57:44.435024 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:44.434987 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:44.435599 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:44.435151 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 15:57:44.435599 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:44.435222 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls podName:649647f7-81ce-4ed1-80d4-7ab2cabda43d nodeName:}" failed. No retries permitted until 2026-06-03 15:57:48.435204598 +0000 UTC m=+41.612174385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls") pod "dns-default-76kk6" (UID: "649647f7-81ce-4ed1-80d4-7ab2cabda43d") : secret "dns-default-metrics-tls" not found Jun 03 15:57:44.536360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:44.536324 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert\") pod \"ingress-canary-l2jxd\" (UID: \"1e3e630c-6304-40da-935c-2dc17eb2ddd0\") " pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:57:44.536522 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:44.536478 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 15:57:44.536561 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:44.536545 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert podName:1e3e630c-6304-40da-935c-2dc17eb2ddd0 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:48.536528037 +0000 UTC m=+41.713497835 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert") pod "ingress-canary-l2jxd" (UID: "1e3e630c-6304-40da-935c-2dc17eb2ddd0") : secret "canary-serving-cert" not found Jun 03 15:57:44.593176 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:44.593090 2576 generic.go:358] "Generic (PLEG): container finished" podID="27386c54-e43c-47d4-89e2-7ae61fb82e65" containerID="d4738314b3a488681d73d23d3f936fb4a8be9d9c6192f544fec82afb43efc912" exitCode=0 Jun 03 15:57:44.593176 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:44.593166 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7xtj" event={"ID":"27386c54-e43c-47d4-89e2-7ae61fb82e65","Type":"ContainerDied","Data":"d4738314b3a488681d73d23d3f936fb4a8be9d9c6192f544fec82afb43efc912"} Jun 03 15:57:45.597604 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:45.597572 2576 generic.go:358] "Generic (PLEG): container finished" podID="27386c54-e43c-47d4-89e2-7ae61fb82e65" containerID="9981f3decca5d48731afad2f00bce40100b0be8546b8f9ddc1864303ab4c141e" exitCode=0 Jun 03 15:57:45.597987 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:45.597624 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7xtj" event={"ID":"27386c54-e43c-47d4-89e2-7ae61fb82e65","Type":"ContainerDied","Data":"9981f3decca5d48731afad2f00bce40100b0be8546b8f9ddc1864303ab4c141e"} Jun 03 15:57:46.603134 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:46.603089 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7xtj" event={"ID":"27386c54-e43c-47d4-89e2-7ae61fb82e65","Type":"ContainerStarted","Data":"a815a9177b68b33da48fbd890b5b00d774bc3f683b983125f700e03f4e95b81d"} Jun 03 15:57:46.624230 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:46.624178 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j7xtj" podStartSLOduration=5.890436593 podStartE2EDuration="39.6241615s" podCreationTimestamp="2026-06-03 15:57:07 +0000 UTC" firstStartedPulling="2026-06-03 15:57:09.982738599 +0000 UTC m=+3.159708400" lastFinishedPulling="2026-06-03 15:57:43.716463504 +0000 UTC m=+36.893433307" observedRunningTime="2026-06-03 15:57:46.622441849 +0000 UTC m=+39.799411657" watchObservedRunningTime="2026-06-03 15:57:46.6241615 +0000 UTC m=+39.801131349" Jun 03 15:57:48.462748 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:48.462705 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:48.463203 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:48.462870 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 15:57:48.463203 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:48.462933 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls podName:649647f7-81ce-4ed1-80d4-7ab2cabda43d nodeName:}" failed. No retries permitted until 2026-06-03 15:57:56.462917908 +0000 UTC m=+49.639887701 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls") pod "dns-default-76kk6" (UID: "649647f7-81ce-4ed1-80d4-7ab2cabda43d") : secret "dns-default-metrics-tls" not found Jun 03 15:57:48.563841 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:48.563806 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert\") pod \"ingress-canary-l2jxd\" (UID: \"1e3e630c-6304-40da-935c-2dc17eb2ddd0\") " pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:57:48.564009 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:48.563956 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 15:57:48.564050 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:48.564022 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert podName:1e3e630c-6304-40da-935c-2dc17eb2ddd0 nodeName:}" failed. No retries permitted until 2026-06-03 15:57:56.564007372 +0000 UTC m=+49.740977159 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert") pod "ingress-canary-l2jxd" (UID: "1e3e630c-6304-40da-935c-2dc17eb2ddd0") : secret "canary-serving-cert" not found Jun 03 15:57:51.185909 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:51.185855 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret\") pod \"global-pull-secret-syncer-zfdzl\" (UID: \"d8f8466e-baef-4149-964b-a5366d798c52\") " pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:51.188570 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:51.188541 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8f8466e-baef-4149-964b-a5366d798c52-original-pull-secret\") pod \"global-pull-secret-syncer-zfdzl\" (UID: \"d8f8466e-baef-4149-964b-a5366d798c52\") " pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:51.400186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:51.400156 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfdzl" Jun 03 15:57:51.525580 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:51.525544 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zfdzl"] Jun 03 15:57:51.529878 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:57:51.529851 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8f8466e_baef_4149_964b_a5366d798c52.slice/crio-f16e203c4c06a83cf5a467c9fa7307c0d1023673037f086493a628289c83aa4d WatchSource:0}: Error finding container f16e203c4c06a83cf5a467c9fa7307c0d1023673037f086493a628289c83aa4d: Status 404 returned error can't find the container with id f16e203c4c06a83cf5a467c9fa7307c0d1023673037f086493a628289c83aa4d Jun 03 15:57:51.612629 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:51.612594 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zfdzl" event={"ID":"d8f8466e-baef-4149-964b-a5366d798c52","Type":"ContainerStarted","Data":"f16e203c4c06a83cf5a467c9fa7307c0d1023673037f086493a628289c83aa4d"} Jun 03 15:57:56.528047 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:56.528008 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:57:56.528497 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:56.528147 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 15:57:56.528497 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:56.528212 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls podName:649647f7-81ce-4ed1-80d4-7ab2cabda43d nodeName:}" failed. No retries permitted until 2026-06-03 15:58:12.528196826 +0000 UTC m=+65.705166613 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls") pod "dns-default-76kk6" (UID: "649647f7-81ce-4ed1-80d4-7ab2cabda43d") : secret "dns-default-metrics-tls" not found Jun 03 15:57:56.624153 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:56.624116 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zfdzl" event={"ID":"d8f8466e-baef-4149-964b-a5366d798c52","Type":"ContainerStarted","Data":"4f6f505e93bb80939c7b33abec5523221a06c88ca87a195ea83d67521754c18a"} Jun 03 15:57:56.628434 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:56.628415 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert\") pod \"ingress-canary-l2jxd\" (UID: \"1e3e630c-6304-40da-935c-2dc17eb2ddd0\") " pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:57:56.628531 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:56.628519 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 15:57:56.628569 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:57:56.628565 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert podName:1e3e630c-6304-40da-935c-2dc17eb2ddd0 nodeName:}" failed. No retries permitted until 2026-06-03 15:58:12.628552474 +0000 UTC m=+65.805522261 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert") pod "ingress-canary-l2jxd" (UID: "1e3e630c-6304-40da-935c-2dc17eb2ddd0") : secret "canary-serving-cert" not found Jun 03 15:57:56.638212 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:57:56.638167 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zfdzl" podStartSLOduration=33.316786709 podStartE2EDuration="37.638153756s" podCreationTimestamp="2026-06-03 15:57:19 +0000 UTC" firstStartedPulling="2026-06-03 15:57:51.531603103 +0000 UTC m=+44.708572892" lastFinishedPulling="2026-06-03 15:57:55.852970149 +0000 UTC m=+49.029939939" observedRunningTime="2026-06-03 15:57:56.63770821 +0000 UTC m=+49.814678019" watchObservedRunningTime="2026-06-03 15:57:56.638153756 +0000 UTC m=+49.815123639" Jun 03 15:58:06.584370 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:06.584339 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bjv7n" Jun 03 15:58:06.776568 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:06.776537 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d8699c84-md6kn"] Jun 03 15:58:06.778780 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:06.778756 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d8699c84-md6kn" Jun 03 15:58:06.781229 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:06.781203 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-swhbd\"" Jun 03 15:58:06.781354 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:06.781321 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 03 15:58:06.781354 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:06.781349 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 03 15:58:06.781446 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:06.781376 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 03 15:58:06.781482 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:06.781461 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 03 15:58:06.788099 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:06.788071 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d8699c84-md6kn"] Jun 03 15:58:06.902131 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:06.902021 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpzdq\" (UniqueName: \"kubernetes.io/projected/ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad-kube-api-access-gpzdq\") pod \"managed-serviceaccount-addon-agent-57d8699c84-md6kn\" (UID: \"ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d8699c84-md6kn" Jun 03 15:58:06.902131 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:06.902104 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57d8699c84-md6kn\" (UID: \"ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d8699c84-md6kn" Jun 03 15:58:07.003352 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:07.003304 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gpzdq\" (UniqueName: \"kubernetes.io/projected/ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad-kube-api-access-gpzdq\") pod \"managed-serviceaccount-addon-agent-57d8699c84-md6kn\" (UID: \"ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d8699c84-md6kn" Jun 03 15:58:07.003425 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:07.003374 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57d8699c84-md6kn\" (UID: \"ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d8699c84-md6kn" Jun 03 15:58:07.005833 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:07.005800 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57d8699c84-md6kn\" (UID: \"ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d8699c84-md6kn" Jun 03 15:58:07.014233 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:07.014208 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpzdq\" (UniqueName: \"kubernetes.io/projected/ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad-kube-api-access-gpzdq\") pod \"managed-serviceaccount-addon-agent-57d8699c84-md6kn\" (UID: \"ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d8699c84-md6kn" Jun 03 15:58:07.099833 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:07.099793 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d8699c84-md6kn" Jun 03 15:58:07.231494 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:07.231464 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d8699c84-md6kn"] Jun 03 15:58:07.235479 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:58:07.235452 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef5f3330_ceb0_4224_86ee_0bcfd2cee9ad.slice/crio-6981b90ac752c07fb0c38cc4712c1052b1089e419abccada06871fb524dd8e96 WatchSource:0}: Error finding container 6981b90ac752c07fb0c38cc4712c1052b1089e419abccada06871fb524dd8e96: Status 404 returned error can't find the container with id 6981b90ac752c07fb0c38cc4712c1052b1089e419abccada06871fb524dd8e96 Jun 03 15:58:07.648377 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:07.648288 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d8699c84-md6kn" event={"ID":"ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad","Type":"ContainerStarted","Data":"6981b90ac752c07fb0c38cc4712c1052b1089e419abccada06871fb524dd8e96"} Jun 03 15:58:10.654588 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:10.654544 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d8699c84-md6kn" event={"ID":"ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad","Type":"ContainerStarted","Data":"045d2fbd9ef9b3da944a512d88778b5e0062ededc341784022a8406ad57b94f5"} Jun 03 15:58:10.669920 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:10.669870 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d8699c84-md6kn" podStartSLOduration=2.249056352 podStartE2EDuration="4.669856172s" podCreationTimestamp="2026-06-03 15:58:06 +0000 UTC" firstStartedPulling="2026-06-03 15:58:07.236859363 +0000 UTC m=+60.413829150" lastFinishedPulling="2026-06-03 15:58:09.657659184 +0000 UTC m=+62.834628970" observedRunningTime="2026-06-03 15:58:10.669350759 +0000 UTC m=+63.846320589" watchObservedRunningTime="2026-06-03 15:58:10.669856172 +0000 UTC m=+63.846825982" Jun 03 15:58:12.542700 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:12.542647 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:58:12.543182 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:12.542827 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 15:58:12.543182 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:12.542895 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls podName:649647f7-81ce-4ed1-80d4-7ab2cabda43d nodeName:}" failed. No retries permitted until 2026-06-03 15:58:44.542877008 +0000 UTC m=+97.719846813 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls") pod "dns-default-76kk6" (UID: "649647f7-81ce-4ed1-80d4-7ab2cabda43d") : secret "dns-default-metrics-tls" not found Jun 03 15:58:12.643028 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:12.642990 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert\") pod \"ingress-canary-l2jxd\" (UID: \"1e3e630c-6304-40da-935c-2dc17eb2ddd0\") " pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:58:12.643156 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:12.643141 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 15:58:12.643210 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:12.643201 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert podName:1e3e630c-6304-40da-935c-2dc17eb2ddd0 nodeName:}" failed. No retries permitted until 2026-06-03 15:58:44.64318351 +0000 UTC m=+97.820153311 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert") pod "ingress-canary-l2jxd" (UID: "1e3e630c-6304-40da-935c-2dc17eb2ddd0") : secret "canary-serving-cert" not found Jun 03 15:58:13.046220 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:13.046173 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs\") pod \"network-metrics-daemon-w5jsj\" (UID: \"2b455f74-b695-4158-8fe3-2fe335bd3e3b\") " pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:58:13.048999 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:13.048977 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 15:58:13.057196 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:13.057172 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 03 15:58:13.057266 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:13.057234 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs podName:2b455f74-b695-4158-8fe3-2fe335bd3e3b nodeName:}" failed. No retries permitted until 2026-06-03 15:59:17.057217237 +0000 UTC m=+130.234187030 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs") pod "network-metrics-daemon-w5jsj" (UID: "2b455f74-b695-4158-8fe3-2fe335bd3e3b") : secret "metrics-daemon-secret" not found Jun 03 15:58:13.146858 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:13.146826 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hn4zm\" (UniqueName: \"kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm\") pod \"network-check-target-g74tp\" (UID: \"5bcd65d8-45ad-42e2-8554-16e218c5b465\") " pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:58:13.149301 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:13.149284 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 15:58:13.159849 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:13.159827 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 15:58:13.170167 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:13.170140 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn4zm\" (UniqueName: \"kubernetes.io/projected/5bcd65d8-45ad-42e2-8554-16e218c5b465-kube-api-access-hn4zm\") pod \"network-check-target-g74tp\" (UID: \"5bcd65d8-45ad-42e2-8554-16e218c5b465\") " pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:58:13.317348 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:13.317265 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xx6f7\"" Jun 03 15:58:13.325144 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:13.325122 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:58:13.445825 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:13.445794 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g74tp"] Jun 03 15:58:13.449504 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:58:13.449483 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bcd65d8_45ad_42e2_8554_16e218c5b465.slice/crio-c92af2c06ec5cab45800266084dcce030852f4a3259b3e0931b0436790f3917c WatchSource:0}: Error finding container c92af2c06ec5cab45800266084dcce030852f4a3259b3e0931b0436790f3917c: Status 404 returned error can't find the container with id c92af2c06ec5cab45800266084dcce030852f4a3259b3e0931b0436790f3917c Jun 03 15:58:13.661547 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:13.661468 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g74tp" event={"ID":"5bcd65d8-45ad-42e2-8554-16e218c5b465","Type":"ContainerStarted","Data":"c92af2c06ec5cab45800266084dcce030852f4a3259b3e0931b0436790f3917c"} Jun 03 15:58:16.668388 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:16.668351 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g74tp" event={"ID":"5bcd65d8-45ad-42e2-8554-16e218c5b465","Type":"ContainerStarted","Data":"12f3bde062de9fedf9da4150bdb27552b79624ee5720ef87322ad076347477da"} Jun 03 15:58:16.668748 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:16.668480 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:58:16.685166 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:16.683722 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-g74tp" podStartSLOduration=67.077147693 podStartE2EDuration="1m9.683705131s" podCreationTimestamp="2026-06-03 15:57:07 +0000 UTC" firstStartedPulling="2026-06-03 15:58:13.453509253 +0000 UTC m=+66.630479040" lastFinishedPulling="2026-06-03 15:58:16.060066688 +0000 UTC m=+69.237036478" observedRunningTime="2026-06-03 15:58:16.682333442 +0000 UTC m=+69.859303253" watchObservedRunningTime="2026-06-03 15:58:16.683705131 +0000 UTC m=+69.860674940" Jun 03 15:58:44.556833 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:44.556752 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:58:44.557263 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:44.556893 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 15:58:44.557263 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:44.556972 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls podName:649647f7-81ce-4ed1-80d4-7ab2cabda43d nodeName:}" failed. No retries permitted until 2026-06-03 15:59:48.55695596 +0000 UTC m=+161.733925748 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls") pod "dns-default-76kk6" (UID: "649647f7-81ce-4ed1-80d4-7ab2cabda43d") : secret "dns-default-metrics-tls" not found Jun 03 15:58:44.657452 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:44.657338 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert\") pod \"ingress-canary-l2jxd\" (UID: \"1e3e630c-6304-40da-935c-2dc17eb2ddd0\") " pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:58:44.657627 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:44.657486 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 15:58:44.657627 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:44.657565 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert podName:1e3e630c-6304-40da-935c-2dc17eb2ddd0 nodeName:}" failed. No retries permitted until 2026-06-03 15:59:48.657545209 +0000 UTC m=+161.834515010 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert") pod "ingress-canary-l2jxd" (UID: "1e3e630c-6304-40da-935c-2dc17eb2ddd0") : secret "canary-serving-cert" not found Jun 03 15:58:47.030540 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.030497 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-f7f955c5f-45jt2"] Jun 03 15:58:47.033176 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.033154 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:47.035526 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.035503 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-9f285\"" Jun 03 15:58:47.035935 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.035918 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 03 15:58:47.036060 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.036041 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 03 15:58:47.036141 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.036121 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 03 15:58:47.036197 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.036141 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 03 15:58:47.036197 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.036155 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 03 15:58:47.036729 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.036714 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 03 15:58:47.046916 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.046888 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-f7f955c5f-45jt2"] Jun 03 15:58:47.071867 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.071840 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-default-certificate\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:47.072048 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.071879 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:47.072048 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.071899 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-stats-auth\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:47.072048 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.071980 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:47.072160 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.072074 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcp8p\" (UniqueName: \"kubernetes.io/projected/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-kube-api-access-rcp8p\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:47.126526 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.126490 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-9npcq"] Jun 03 15:58:47.128432 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.128411 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-9npcq" Jun 03 15:58:47.131066 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.131037 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-nw98n\"" Jun 03 15:58:47.131167 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.131116 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 03 15:58:47.131222 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.131117 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 03 15:58:47.140103 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.140080 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-9npcq"] Jun 03 15:58:47.173191 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.173161 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-stats-auth\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:47.173344 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.173209 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:47.173344 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:47.173288 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 03 15:58:47.173344 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.173326 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvclz\" (UniqueName: \"kubernetes.io/projected/d0510938-d2f4-4c49-a674-b893977e8443-kube-api-access-cvclz\") pod \"volume-data-source-validator-7598fd5bd-9npcq\" (UID: \"d0510938-d2f4-4c49-a674-b893977e8443\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-9npcq" Jun 03 15:58:47.173473 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:47.173351 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs podName:3fae3b07-0b61-4b4c-84e3-c6203b79c4af nodeName:}" failed. No retries permitted until 2026-06-03 15:58:47.673335513 +0000 UTC m=+100.850305300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs") pod "router-default-f7f955c5f-45jt2" (UID: "3fae3b07-0b61-4b4c-84e3-c6203b79c4af") : secret "router-metrics-certs-default" not found Jun 03 15:58:47.173473 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.173456 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rcp8p\" (UniqueName: \"kubernetes.io/projected/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-kube-api-access-rcp8p\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:47.173561 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.173490 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-default-certificate\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:47.173561 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.173530 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:47.173651 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:47.173641 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle podName:3fae3b07-0b61-4b4c-84e3-c6203b79c4af nodeName:}" failed. No retries permitted until 2026-06-03 15:58:47.67362672 +0000 UTC m=+100.850596521 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle") pod "router-default-f7f955c5f-45jt2" (UID: "3fae3b07-0b61-4b4c-84e3-c6203b79c4af") : configmap references non-existent config key: service-ca.crt Jun 03 15:58:47.175593 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.175561 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-stats-auth\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:47.175833 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.175814 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-default-certificate\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:47.183865 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.183843 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcp8p\" (UniqueName: \"kubernetes.io/projected/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-kube-api-access-rcp8p\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:47.274022 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.273984 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cvclz\" (UniqueName: \"kubernetes.io/projected/d0510938-d2f4-4c49-a674-b893977e8443-kube-api-access-cvclz\") pod \"volume-data-source-validator-7598fd5bd-9npcq\" (UID: \"d0510938-d2f4-4c49-a674-b893977e8443\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-9npcq" Jun 03 15:58:47.283023 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.282963 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvclz\" (UniqueName: \"kubernetes.io/projected/d0510938-d2f4-4c49-a674-b893977e8443-kube-api-access-cvclz\") pod \"volume-data-source-validator-7598fd5bd-9npcq\" (UID: \"d0510938-d2f4-4c49-a674-b893977e8443\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-9npcq" Jun 03 15:58:47.438623 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.438567 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-9npcq" Jun 03 15:58:47.558005 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.557926 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-9npcq"] Jun 03 15:58:47.561250 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:58:47.561221 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0510938_d2f4_4c49_a674_b893977e8443.slice/crio-4c980450a709af9e46666d7d78411ef547e4cee50c133753835b6609acd60ddb WatchSource:0}: Error finding container 4c980450a709af9e46666d7d78411ef547e4cee50c133753835b6609acd60ddb: Status 404 returned error can't find the container with id 4c980450a709af9e46666d7d78411ef547e4cee50c133753835b6609acd60ddb Jun 03 15:58:47.672555 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.672523 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-g74tp" Jun 03 15:58:47.676659 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.676635 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:47.676804 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.676696 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:47.676879 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:47.676808 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 03 15:58:47.676879 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:47.676818 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle podName:3fae3b07-0b61-4b4c-84e3-c6203b79c4af nodeName:}" failed. No retries permitted until 2026-06-03 15:58:48.676802383 +0000 UTC m=+101.853772173 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle") pod "router-default-f7f955c5f-45jt2" (UID: "3fae3b07-0b61-4b4c-84e3-c6203b79c4af") : configmap references non-existent config key: service-ca.crt Jun 03 15:58:47.676964 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:47.676882 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs podName:3fae3b07-0b61-4b4c-84e3-c6203b79c4af nodeName:}" failed. No retries permitted until 2026-06-03 15:58:48.676864464 +0000 UTC m=+101.853834265 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs") pod "router-default-f7f955c5f-45jt2" (UID: "3fae3b07-0b61-4b4c-84e3-c6203b79c4af") : secret "router-metrics-certs-default" not found Jun 03 15:58:47.731212 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:47.731179 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-9npcq" event={"ID":"d0510938-d2f4-4c49-a674-b893977e8443","Type":"ContainerStarted","Data":"4c980450a709af9e46666d7d78411ef547e4cee50c133753835b6609acd60ddb"} Jun 03 15:58:48.682525 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:48.682487 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:48.682937 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:48.682541 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:48.682937 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:48.682657 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle podName:3fae3b07-0b61-4b4c-84e3-c6203b79c4af nodeName:}" failed. No retries permitted until 2026-06-03 15:58:50.68264052 +0000 UTC m=+103.859610310 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle") pod "router-default-f7f955c5f-45jt2" (UID: "3fae3b07-0b61-4b4c-84e3-c6203b79c4af") : configmap references non-existent config key: service-ca.crt Jun 03 15:58:48.682937 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:48.682693 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 03 15:58:48.682937 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:48.682735 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs podName:3fae3b07-0b61-4b4c-84e3-c6203b79c4af nodeName:}" failed. No retries permitted until 2026-06-03 15:58:50.682723475 +0000 UTC m=+103.859693270 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs") pod "router-default-f7f955c5f-45jt2" (UID: "3fae3b07-0b61-4b4c-84e3-c6203b79c4af") : secret "router-metrics-certs-default" not found Jun 03 15:58:49.736132 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:49.736095 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-9npcq" event={"ID":"d0510938-d2f4-4c49-a674-b893977e8443","Type":"ContainerStarted","Data":"12a7cf8f54d7b3a7d0e64f89944a476cffbd5b11a59222e28a483b361f805a42"} Jun 03 15:58:49.750844 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:49.750785 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-9npcq" podStartSLOduration=1.267347244 podStartE2EDuration="2.750751279s" podCreationTimestamp="2026-06-03 15:58:47 +0000 UTC" firstStartedPulling="2026-06-03 15:58:47.562902329 +0000 UTC m=+100.739872115" lastFinishedPulling="2026-06-03 15:58:49.04630636 +0000 UTC m=+102.223276150" observedRunningTime="2026-06-03 15:58:49.75016171 +0000 UTC m=+102.927131518" watchObservedRunningTime="2026-06-03 15:58:49.750751279 +0000 UTC m=+102.927721084" Jun 03 15:58:50.698582 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:50.698531 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:50.698799 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:50.698630 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:50.698799 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:50.698711 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 03 15:58:50.698799 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:50.698761 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle podName:3fae3b07-0b61-4b4c-84e3-c6203b79c4af nodeName:}" failed. No retries permitted until 2026-06-03 15:58:54.69874278 +0000 UTC m=+107.875712580 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle") pod "router-default-f7f955c5f-45jt2" (UID: "3fae3b07-0b61-4b4c-84e3-c6203b79c4af") : configmap references non-existent config key: service-ca.crt Jun 03 15:58:50.698951 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:50.698815 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs podName:3fae3b07-0b61-4b4c-84e3-c6203b79c4af nodeName:}" failed. No retries permitted until 2026-06-03 15:58:54.698803774 +0000 UTC m=+107.875773566 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs") pod "router-default-f7f955c5f-45jt2" (UID: "3fae3b07-0b61-4b4c-84e3-c6203b79c4af") : secret "router-metrics-certs-default" not found Jun 03 15:58:52.012515 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.012482 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv"] Jun 03 15:58:52.014505 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.014484 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv" Jun 03 15:58:52.016979 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.016959 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 03 15:58:52.017877 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.017857 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-rzx2n\"" Jun 03 15:58:52.017877 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.017868 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 03 15:58:52.018024 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.017888 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 03 15:58:52.018024 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.017891 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 03 15:58:52.023968 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.023939 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv"] Jun 03 15:58:52.107795 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.107730 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6ec5c06b-c6ab-4826-9076-93046a57fb73-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-vbrqv\" (UID: \"6ec5c06b-c6ab-4826-9076-93046a57fb73\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv" Jun 03 15:58:52.107986 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.107809 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g4nw\" (UniqueName: \"kubernetes.io/projected/6ec5c06b-c6ab-4826-9076-93046a57fb73-kube-api-access-9g4nw\") pod \"kube-storage-version-migrator-operator-96ddc787c-vbrqv\" (UID: \"6ec5c06b-c6ab-4826-9076-93046a57fb73\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv" Jun 03 15:58:52.107986 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.107859 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6ec5c06b-c6ab-4826-9076-93046a57fb73-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-vbrqv\" (UID: \"6ec5c06b-c6ab-4826-9076-93046a57fb73\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv" Jun 03 15:58:52.208462 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.208421 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6ec5c06b-c6ab-4826-9076-93046a57fb73-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-vbrqv\" (UID: \"6ec5c06b-c6ab-4826-9076-93046a57fb73\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv" Jun 03 15:58:52.208658 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.208486 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9g4nw\" (UniqueName: \"kubernetes.io/projected/6ec5c06b-c6ab-4826-9076-93046a57fb73-kube-api-access-9g4nw\") pod \"kube-storage-version-migrator-operator-96ddc787c-vbrqv\" (UID: \"6ec5c06b-c6ab-4826-9076-93046a57fb73\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv" Jun 03 15:58:52.208658 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.208519 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6ec5c06b-c6ab-4826-9076-93046a57fb73-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-vbrqv\" (UID: \"6ec5c06b-c6ab-4826-9076-93046a57fb73\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv" Jun 03 15:58:52.209127 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.209103 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6ec5c06b-c6ab-4826-9076-93046a57fb73-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-vbrqv\" (UID: \"6ec5c06b-c6ab-4826-9076-93046a57fb73\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv" Jun 03 15:58:52.210783 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.210750 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6ec5c06b-c6ab-4826-9076-93046a57fb73-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-vbrqv\" (UID: \"6ec5c06b-c6ab-4826-9076-93046a57fb73\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv" Jun 03 15:58:52.216291 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.216265 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g4nw\" (UniqueName: \"kubernetes.io/projected/6ec5c06b-c6ab-4826-9076-93046a57fb73-kube-api-access-9g4nw\") pod \"kube-storage-version-migrator-operator-96ddc787c-vbrqv\" (UID: \"6ec5c06b-c6ab-4826-9076-93046a57fb73\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv" Jun 03 15:58:52.323738 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.323639 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv" Jun 03 15:58:52.448882 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.448853 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv"] Jun 03 15:58:52.452402 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:58:52.452357 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ec5c06b_c6ab_4826_9076_93046a57fb73.slice/crio-140264ce2bdc9fb1c9816fbd7e182302b33b3ff514b923a2a2d6fa8b960bef7a WatchSource:0}: Error finding container 140264ce2bdc9fb1c9816fbd7e182302b33b3ff514b923a2a2d6fa8b960bef7a: Status 404 returned error can't find the container with id 140264ce2bdc9fb1c9816fbd7e182302b33b3ff514b923a2a2d6fa8b960bef7a Jun 03 15:58:52.742996 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:52.742955 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv" event={"ID":"6ec5c06b-c6ab-4826-9076-93046a57fb73","Type":"ContainerStarted","Data":"140264ce2bdc9fb1c9816fbd7e182302b33b3ff514b923a2a2d6fa8b960bef7a"} Jun 03 15:58:53.577524 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:53.577499 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q5cmz_16c438af-88c9-4b06-be30-c984a697afeb/dns-node-resolver/0.log" Jun 03 15:58:53.978005 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:53.977982 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4ppxs_0aa350a0-203c-4286-b184-d62efbd0af97/node-ca/0.log" Jun 03 15:58:54.614695 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:54.614657 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-5ctz2"] Jun 03 15:58:54.617250 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:54.617227 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-5ctz2" Jun 03 15:58:54.619569 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:54.619543 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-xcph8\"" Jun 03 15:58:54.626095 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:54.626070 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-5ctz2"] Jun 03 15:58:54.727368 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:54.727326 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:54.727568 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:54.727510 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:58:54.727568 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:54.727524 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle podName:3fae3b07-0b61-4b4c-84e3-c6203b79c4af nodeName:}" failed. No retries permitted until 2026-06-03 15:59:02.727502168 +0000 UTC m=+115.904471958 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle") pod "router-default-f7f955c5f-45jt2" (UID: "3fae3b07-0b61-4b4c-84e3-c6203b79c4af") : configmap references non-existent config key: service-ca.crt Jun 03 15:58:54.727691 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:54.727623 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 03 15:58:54.727691 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:54.727637 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h46jz\" (UniqueName: \"kubernetes.io/projected/d4445224-20e9-4308-bdcd-b1255e1fee20-kube-api-access-h46jz\") pod \"network-check-source-56d84b9f44-5ctz2\" (UID: \"d4445224-20e9-4308-bdcd-b1255e1fee20\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-5ctz2" Jun 03 15:58:54.727691 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:58:54.727677 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs podName:3fae3b07-0b61-4b4c-84e3-c6203b79c4af nodeName:}" failed. No retries permitted until 2026-06-03 15:59:02.72766079 +0000 UTC m=+115.904630595 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs") pod "router-default-f7f955c5f-45jt2" (UID: "3fae3b07-0b61-4b4c-84e3-c6203b79c4af") : secret "router-metrics-certs-default" not found Jun 03 15:58:54.828165 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:54.828117 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h46jz\" (UniqueName: \"kubernetes.io/projected/d4445224-20e9-4308-bdcd-b1255e1fee20-kube-api-access-h46jz\") pod \"network-check-source-56d84b9f44-5ctz2\" (UID: \"d4445224-20e9-4308-bdcd-b1255e1fee20\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-5ctz2" Jun 03 15:58:54.837069 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:54.837009 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h46jz\" (UniqueName: \"kubernetes.io/projected/d4445224-20e9-4308-bdcd-b1255e1fee20-kube-api-access-h46jz\") pod \"network-check-source-56d84b9f44-5ctz2\" (UID: \"d4445224-20e9-4308-bdcd-b1255e1fee20\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-5ctz2" Jun 03 15:58:54.927953 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:54.927859 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-5ctz2" Jun 03 15:58:55.137612 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:55.137565 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-5ctz2"] Jun 03 15:58:55.141566 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:58:55.141524 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4445224_20e9_4308_bdcd_b1255e1fee20.slice/crio-20398816d39256c4b014ac81b8d4918beb7c4bbd5b14d94db8033286d9c0b8a7 WatchSource:0}: Error finding container 20398816d39256c4b014ac81b8d4918beb7c4bbd5b14d94db8033286d9c0b8a7: Status 404 returned error can't find the container with id 20398816d39256c4b014ac81b8d4918beb7c4bbd5b14d94db8033286d9c0b8a7 Jun 03 15:58:55.751288 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:55.751249 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-5ctz2" event={"ID":"d4445224-20e9-4308-bdcd-b1255e1fee20","Type":"ContainerStarted","Data":"0b90a8fbd573ca5a41e9c753d8703b4621512803f7cdf66785e1fbc2dc188bf0"} Jun 03 15:58:55.751725 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:55.751295 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-5ctz2" event={"ID":"d4445224-20e9-4308-bdcd-b1255e1fee20","Type":"ContainerStarted","Data":"20398816d39256c4b014ac81b8d4918beb7c4bbd5b14d94db8033286d9c0b8a7"} Jun 03 15:58:55.752628 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:55.752600 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv" event={"ID":"6ec5c06b-c6ab-4826-9076-93046a57fb73","Type":"ContainerStarted","Data":"404b728fcd1d2c95a568d2a6462bce12f18babf334b0a5120250daaa6e75efae"} Jun 03 15:58:55.765944 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:55.765894 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-5ctz2" podStartSLOduration=1.765878566 podStartE2EDuration="1.765878566s" podCreationTimestamp="2026-06-03 15:58:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 15:58:55.765152767 +0000 UTC m=+108.942122576" watchObservedRunningTime="2026-06-03 15:58:55.765878566 +0000 UTC m=+108.942848376" Jun 03 15:58:55.779907 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:58:55.779847 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv" podStartSLOduration=2.190470376 podStartE2EDuration="4.779825058s" podCreationTimestamp="2026-06-03 15:58:51 +0000 UTC" firstStartedPulling="2026-06-03 15:58:52.454259223 +0000 UTC m=+105.631229010" lastFinishedPulling="2026-06-03 15:58:55.043613902 +0000 UTC m=+108.220583692" observedRunningTime="2026-06-03 15:58:55.77936219 +0000 UTC m=+108.956332000" watchObservedRunningTime="2026-06-03 15:58:55.779825058 +0000 UTC m=+108.956794868" Jun 03 15:59:00.742854 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.742811 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-685df94d57-zzxsx"] Jun 03 15:59:00.744983 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.744966 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-zzxsx" Jun 03 15:59:00.747270 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.747245 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-p9shg\"" Jun 03 15:59:00.747406 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.747269 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 03 15:59:00.747406 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.747290 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 03 15:59:00.748210 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.748193 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 03 15:59:00.748298 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.748227 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 03 15:59:00.755972 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.755943 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-zzxsx"] Jun 03 15:59:00.871166 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.871111 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f1114a3a-86c1-4af0-a3f2-b27947d5584a-signing-key\") pod \"service-ca-685df94d57-zzxsx\" (UID: \"f1114a3a-86c1-4af0-a3f2-b27947d5584a\") " pod="openshift-service-ca/service-ca-685df94d57-zzxsx" Jun 03 15:59:00.871325 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.871193 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f1114a3a-86c1-4af0-a3f2-b27947d5584a-signing-cabundle\") pod \"service-ca-685df94d57-zzxsx\" (UID: \"f1114a3a-86c1-4af0-a3f2-b27947d5584a\") " pod="openshift-service-ca/service-ca-685df94d57-zzxsx" Jun 03 15:59:00.871325 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.871286 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49wg2\" (UniqueName: \"kubernetes.io/projected/f1114a3a-86c1-4af0-a3f2-b27947d5584a-kube-api-access-49wg2\") pod \"service-ca-685df94d57-zzxsx\" (UID: \"f1114a3a-86c1-4af0-a3f2-b27947d5584a\") " pod="openshift-service-ca/service-ca-685df94d57-zzxsx" Jun 03 15:59:00.971850 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.971796 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-49wg2\" (UniqueName: \"kubernetes.io/projected/f1114a3a-86c1-4af0-a3f2-b27947d5584a-kube-api-access-49wg2\") pod \"service-ca-685df94d57-zzxsx\" (UID: \"f1114a3a-86c1-4af0-a3f2-b27947d5584a\") " pod="openshift-service-ca/service-ca-685df94d57-zzxsx" Jun 03 15:59:00.971945 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.971868 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f1114a3a-86c1-4af0-a3f2-b27947d5584a-signing-key\") pod \"service-ca-685df94d57-zzxsx\" (UID: \"f1114a3a-86c1-4af0-a3f2-b27947d5584a\") " pod="openshift-service-ca/service-ca-685df94d57-zzxsx" Jun 03 15:59:00.971945 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.971913 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f1114a3a-86c1-4af0-a3f2-b27947d5584a-signing-cabundle\") pod \"service-ca-685df94d57-zzxsx\" (UID: \"f1114a3a-86c1-4af0-a3f2-b27947d5584a\") " pod="openshift-service-ca/service-ca-685df94d57-zzxsx" Jun 03 15:59:00.972524 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.972497 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f1114a3a-86c1-4af0-a3f2-b27947d5584a-signing-cabundle\") pod \"service-ca-685df94d57-zzxsx\" (UID: \"f1114a3a-86c1-4af0-a3f2-b27947d5584a\") " pod="openshift-service-ca/service-ca-685df94d57-zzxsx" Jun 03 15:59:00.974257 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.974234 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f1114a3a-86c1-4af0-a3f2-b27947d5584a-signing-key\") pod \"service-ca-685df94d57-zzxsx\" (UID: \"f1114a3a-86c1-4af0-a3f2-b27947d5584a\") " pod="openshift-service-ca/service-ca-685df94d57-zzxsx" Jun 03 15:59:00.979610 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:00.979585 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49wg2\" (UniqueName: \"kubernetes.io/projected/f1114a3a-86c1-4af0-a3f2-b27947d5584a-kube-api-access-49wg2\") pod \"service-ca-685df94d57-zzxsx\" (UID: \"f1114a3a-86c1-4af0-a3f2-b27947d5584a\") " pod="openshift-service-ca/service-ca-685df94d57-zzxsx" Jun 03 15:59:01.054217 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:01.054132 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-zzxsx" Jun 03 15:59:01.177474 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:01.177440 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-zzxsx"] Jun 03 15:59:01.180840 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:59:01.180811 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1114a3a_86c1_4af0_a3f2_b27947d5584a.slice/crio-6a39ae91bbbfd1b56a91d6b0346e83f6e621422b8bfd19a50fa7919fc02db628 WatchSource:0}: Error finding container 6a39ae91bbbfd1b56a91d6b0346e83f6e621422b8bfd19a50fa7919fc02db628: Status 404 returned error can't find the container with id 6a39ae91bbbfd1b56a91d6b0346e83f6e621422b8bfd19a50fa7919fc02db628 Jun 03 15:59:01.769188 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:01.769136 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-zzxsx" event={"ID":"f1114a3a-86c1-4af0-a3f2-b27947d5584a","Type":"ContainerStarted","Data":"6a39ae91bbbfd1b56a91d6b0346e83f6e621422b8bfd19a50fa7919fc02db628"} Jun 03 15:59:02.786512 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:02.786469 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:59:02.787017 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:02.786547 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:59:02.787017 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:59:02.786655 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle podName:3fae3b07-0b61-4b4c-84e3-c6203b79c4af nodeName:}" failed. No retries permitted until 2026-06-03 15:59:18.786634826 +0000 UTC m=+131.963604627 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle") pod "router-default-f7f955c5f-45jt2" (UID: "3fae3b07-0b61-4b4c-84e3-c6203b79c4af") : configmap references non-existent config key: service-ca.crt Jun 03 15:59:02.787017 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:59:02.786696 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 03 15:59:02.787017 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:59:02.786778 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs podName:3fae3b07-0b61-4b4c-84e3-c6203b79c4af nodeName:}" failed. No retries permitted until 2026-06-03 15:59:18.786752977 +0000 UTC m=+131.963722764 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs") pod "router-default-f7f955c5f-45jt2" (UID: "3fae3b07-0b61-4b4c-84e3-c6203b79c4af") : secret "router-metrics-certs-default" not found Jun 03 15:59:03.775110 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:03.775070 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-zzxsx" event={"ID":"f1114a3a-86c1-4af0-a3f2-b27947d5584a","Type":"ContainerStarted","Data":"48a1efb1fefd9ba1fae0efdfa613e63a3c20564c13cdaf31871f596b3365b5f0"} Jun 03 15:59:03.793163 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:03.793111 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-685df94d57-zzxsx" podStartSLOduration=1.862955539 podStartE2EDuration="3.793093211s" podCreationTimestamp="2026-06-03 15:59:00 +0000 UTC" firstStartedPulling="2026-06-03 15:59:01.182549847 +0000 UTC m=+114.359519638" lastFinishedPulling="2026-06-03 15:59:03.11268752 +0000 UTC m=+116.289657310" observedRunningTime="2026-06-03 15:59:03.79276246 +0000 UTC m=+116.969732271" watchObservedRunningTime="2026-06-03 15:59:03.793093211 +0000 UTC m=+116.970063054" Jun 03 15:59:17.101035 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:17.100996 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs\") pod \"network-metrics-daemon-w5jsj\" (UID: \"2b455f74-b695-4158-8fe3-2fe335bd3e3b\") " pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:59:17.103376 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:17.103351 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b455f74-b695-4158-8fe3-2fe335bd3e3b-metrics-certs\") pod \"network-metrics-daemon-w5jsj\" (UID: \"2b455f74-b695-4158-8fe3-2fe335bd3e3b\") " pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:59:17.211420 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:17.211386 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qk87p\"" Jun 03 15:59:17.220155 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:17.220132 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5jsj" Jun 03 15:59:17.340267 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:17.340237 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w5jsj"] Jun 03 15:59:17.343343 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:59:17.343320 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b455f74_b695_4158_8fe3_2fe335bd3e3b.slice/crio-1864cb8822c5b644e23d7326a785e7a04a5adc0e861099dbb49b80a7c2a8f9b6 WatchSource:0}: Error finding container 1864cb8822c5b644e23d7326a785e7a04a5adc0e861099dbb49b80a7c2a8f9b6: Status 404 returned error can't find the container with id 1864cb8822c5b644e23d7326a785e7a04a5adc0e861099dbb49b80a7c2a8f9b6 Jun 03 15:59:17.810899 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:17.810855 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w5jsj" event={"ID":"2b455f74-b695-4158-8fe3-2fe335bd3e3b","Type":"ContainerStarted","Data":"1864cb8822c5b644e23d7326a785e7a04a5adc0e861099dbb49b80a7c2a8f9b6"} Jun 03 15:59:18.815654 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:18.815624 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:59:18.816172 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:18.815682 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:59:18.816364 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:18.816335 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-service-ca-bundle\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:59:18.817362 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:18.817336 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w5jsj" event={"ID":"2b455f74-b695-4158-8fe3-2fe335bd3e3b","Type":"ContainerStarted","Data":"d2429ec68447ddd3ec52c93b6546dda4ab8180b2c45bc4a3bd8d99bddcb9dd14"} Jun 03 15:59:18.817443 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:18.817371 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w5jsj" event={"ID":"2b455f74-b695-4158-8fe3-2fe335bd3e3b","Type":"ContainerStarted","Data":"f8f7c6711db3e9cdc78afafdcb077206233f246c6b45e7b5b8cabc42ffbf2815"} Jun 03 15:59:18.818300 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:18.818284 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fae3b07-0b61-4b4c-84e3-c6203b79c4af-metrics-certs\") pod \"router-default-f7f955c5f-45jt2\" (UID: \"3fae3b07-0b61-4b4c-84e3-c6203b79c4af\") " pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:59:18.834563 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:18.834514 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-w5jsj" podStartSLOduration=130.945545377 podStartE2EDuration="2m11.834498523s" podCreationTimestamp="2026-06-03 15:57:07 +0000 UTC" firstStartedPulling="2026-06-03 15:59:17.345014871 +0000 UTC m=+130.521984659" lastFinishedPulling="2026-06-03 15:59:18.233968015 +0000 UTC m=+131.410937805" observedRunningTime="2026-06-03 15:59:18.83397381 +0000 UTC m=+132.010943620" watchObservedRunningTime="2026-06-03 15:59:18.834498523 +0000 UTC m=+132.011468333" Jun 03 15:59:18.844312 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:18.844283 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-9f285\"" Jun 03 15:59:18.852255 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:18.852216 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:59:18.973933 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:18.973891 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-f7f955c5f-45jt2"] Jun 03 15:59:18.975915 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:59:18.975886 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fae3b07_0b61_4b4c_84e3_c6203b79c4af.slice/crio-75a82af34d9cd09daff5397d2b76dcf45b251dca4d8e70e19d0e8387db7a0a27 WatchSource:0}: Error finding container 75a82af34d9cd09daff5397d2b76dcf45b251dca4d8e70e19d0e8387db7a0a27: Status 404 returned error can't find the container with id 75a82af34d9cd09daff5397d2b76dcf45b251dca4d8e70e19d0e8387db7a0a27 Jun 03 15:59:19.821630 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:19.821589 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-f7f955c5f-45jt2" event={"ID":"3fae3b07-0b61-4b4c-84e3-c6203b79c4af","Type":"ContainerStarted","Data":"93a4d91601f89da0b52dcacbd343d08cff8039bcbc88f774f73e287bf9a14cbb"} Jun 03 15:59:19.821630 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:19.821631 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-f7f955c5f-45jt2" event={"ID":"3fae3b07-0b61-4b4c-84e3-c6203b79c4af","Type":"ContainerStarted","Data":"75a82af34d9cd09daff5397d2b76dcf45b251dca4d8e70e19d0e8387db7a0a27"} Jun 03 15:59:19.839796 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:19.839716 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-f7f955c5f-45jt2" podStartSLOduration=32.839699261 podStartE2EDuration="32.839699261s" podCreationTimestamp="2026-06-03 15:58:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 15:59:19.839358188 +0000 UTC m=+133.016327998" watchObservedRunningTime="2026-06-03 15:59:19.839699261 +0000 UTC m=+133.016669071" Jun 03 15:59:19.852655 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:19.852609 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:59:19.855258 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:19.855238 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:59:20.824267 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:20.824235 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:59:20.825554 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:20.825532 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-f7f955c5f-45jt2" Jun 03 15:59:23.175346 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.175314 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-k7xdg"] Jun 03 15:59:23.177252 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.177233 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k7xdg" Jun 03 15:59:23.181644 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.181619 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 03 15:59:23.181801 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.181643 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wl95k\"" Jun 03 15:59:23.181801 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.181619 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 03 15:59:23.187463 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.187436 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-k7xdg"] Jun 03 15:59:23.250828 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.250780 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f6ff88f-c358-44b6-aa8e-ee0b2f094036-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-k7xdg\" (UID: \"4f6ff88f-c358-44b6-aa8e-ee0b2f094036\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k7xdg" Jun 03 15:59:23.251003 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.250900 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4f6ff88f-c358-44b6-aa8e-ee0b2f094036-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-k7xdg\" (UID: \"4f6ff88f-c358-44b6-aa8e-ee0b2f094036\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k7xdg" Jun 03 15:59:23.286277 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.286242 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-nchxs"] Jun 03 15:59:23.288256 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.288229 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-nchxs" Jun 03 15:59:23.291456 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.291430 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-x97pm\"" Jun 03 15:59:23.291858 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.291832 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 03 15:59:23.291943 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.291900 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kfhg4"] Jun 03 15:59:23.293934 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.293901 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-658b55885b-dt9vz"] Jun 03 15:59:23.294099 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.294079 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.297093 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.297070 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 03 15:59:23.299951 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.298962 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 03 15:59:23.299951 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.299290 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 03 15:59:23.299951 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.299874 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 03 15:59:23.301206 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.301175 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-25cmw\"" Jun 03 15:59:23.302865 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.302847 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.305125 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.305106 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 03 15:59:23.306216 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.306059 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 03 15:59:23.306945 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.306913 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2c4jd\"" Jun 03 15:59:23.307127 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.307111 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 03 15:59:23.307472 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.307455 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-nchxs"] Jun 03 15:59:23.312496 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.312470 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 03 15:59:23.316651 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.316139 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kfhg4"] Jun 03 15:59:23.316651 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.316643 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-658b55885b-dt9vz"] Jun 03 15:59:23.351403 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.351366 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f6ff88f-c358-44b6-aa8e-ee0b2f094036-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-k7xdg\" (UID: \"4f6ff88f-c358-44b6-aa8e-ee0b2f094036\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k7xdg" Jun 03 15:59:23.351568 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.351415 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4f6ff88f-c358-44b6-aa8e-ee0b2f094036-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-k7xdg\" (UID: \"4f6ff88f-c358-44b6-aa8e-ee0b2f094036\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k7xdg" Jun 03 15:59:23.352092 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.352062 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4f6ff88f-c358-44b6-aa8e-ee0b2f094036-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-k7xdg\" (UID: \"4f6ff88f-c358-44b6-aa8e-ee0b2f094036\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k7xdg" Jun 03 15:59:23.353733 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.353712 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f6ff88f-c358-44b6-aa8e-ee0b2f094036-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-k7xdg\" (UID: \"4f6ff88f-c358-44b6-aa8e-ee0b2f094036\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k7xdg" Jun 03 15:59:23.452756 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.452726 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g5dx\" (UniqueName: \"kubernetes.io/projected/05a59322-0f9f-427a-8cfc-2eeb363397f5-kube-api-access-9g5dx\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.452930 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.452761 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05a59322-0f9f-427a-8cfc-2eeb363397f5-installation-pull-secrets\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.452930 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.452851 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05a59322-0f9f-427a-8cfc-2eeb363397f5-bound-sa-token\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.452930 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.452883 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05a59322-0f9f-427a-8cfc-2eeb363397f5-registry-certificates\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.452930 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.452903 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d894be86-2b42-4354-aebe-c3533846e9dd-crio-socket\") pod \"insights-runtime-extractor-kfhg4\" (UID: \"d894be86-2b42-4354-aebe-c3533846e9dd\") " pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.452930 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.452925 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05a59322-0f9f-427a-8cfc-2eeb363397f5-trusted-ca\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.453093 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.453000 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05a59322-0f9f-427a-8cfc-2eeb363397f5-ca-trust-extracted\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.453093 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.453026 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d894be86-2b42-4354-aebe-c3533846e9dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kfhg4\" (UID: \"d894be86-2b42-4354-aebe-c3533846e9dd\") " pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.453093 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.453081 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/86fac976-acfe-4419-9e2a-0f468bed1f6d-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-nchxs\" (UID: \"86fac976-acfe-4419-9e2a-0f468bed1f6d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-nchxs" Jun 03 15:59:23.453183 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.453106 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05a59322-0f9f-427a-8cfc-2eeb363397f5-registry-tls\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.453183 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.453123 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d894be86-2b42-4354-aebe-c3533846e9dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kfhg4\" (UID: \"d894be86-2b42-4354-aebe-c3533846e9dd\") " pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.453183 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.453139 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d894be86-2b42-4354-aebe-c3533846e9dd-data-volume\") pod \"insights-runtime-extractor-kfhg4\" (UID: \"d894be86-2b42-4354-aebe-c3533846e9dd\") " pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.453183 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.453161 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05a59322-0f9f-427a-8cfc-2eeb363397f5-image-registry-private-configuration\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.453299 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.453211 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-674pk\" (UniqueName: \"kubernetes.io/projected/d894be86-2b42-4354-aebe-c3533846e9dd-kube-api-access-674pk\") pod \"insights-runtime-extractor-kfhg4\" (UID: \"d894be86-2b42-4354-aebe-c3533846e9dd\") " pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.487462 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.487429 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k7xdg" Jun 03 15:59:23.559354 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.559251 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-674pk\" (UniqueName: \"kubernetes.io/projected/d894be86-2b42-4354-aebe-c3533846e9dd-kube-api-access-674pk\") pod \"insights-runtime-extractor-kfhg4\" (UID: \"d894be86-2b42-4354-aebe-c3533846e9dd\") " pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.559354 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.559319 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9g5dx\" (UniqueName: \"kubernetes.io/projected/05a59322-0f9f-427a-8cfc-2eeb363397f5-kube-api-access-9g5dx\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.559354 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.559352 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05a59322-0f9f-427a-8cfc-2eeb363397f5-installation-pull-secrets\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.559634 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.559385 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05a59322-0f9f-427a-8cfc-2eeb363397f5-bound-sa-token\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.559634 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.559413 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05a59322-0f9f-427a-8cfc-2eeb363397f5-registry-certificates\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.559634 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.559444 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d894be86-2b42-4354-aebe-c3533846e9dd-crio-socket\") pod \"insights-runtime-extractor-kfhg4\" (UID: \"d894be86-2b42-4354-aebe-c3533846e9dd\") " pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.559634 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.559475 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05a59322-0f9f-427a-8cfc-2eeb363397f5-trusted-ca\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.559634 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.559509 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05a59322-0f9f-427a-8cfc-2eeb363397f5-ca-trust-extracted\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.559634 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.559532 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d894be86-2b42-4354-aebe-c3533846e9dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kfhg4\" (UID: \"d894be86-2b42-4354-aebe-c3533846e9dd\") " pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.559634 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.559600 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/86fac976-acfe-4419-9e2a-0f468bed1f6d-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-nchxs\" (UID: \"86fac976-acfe-4419-9e2a-0f468bed1f6d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-nchxs" Jun 03 15:59:23.559634 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.559629 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05a59322-0f9f-427a-8cfc-2eeb363397f5-registry-tls\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.560084 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.559653 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d894be86-2b42-4354-aebe-c3533846e9dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kfhg4\" (UID: \"d894be86-2b42-4354-aebe-c3533846e9dd\") " pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.560084 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.559679 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d894be86-2b42-4354-aebe-c3533846e9dd-data-volume\") pod \"insights-runtime-extractor-kfhg4\" (UID: \"d894be86-2b42-4354-aebe-c3533846e9dd\") " pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.560084 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.559704 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05a59322-0f9f-427a-8cfc-2eeb363397f5-image-registry-private-configuration\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.561324 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.561293 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05a59322-0f9f-427a-8cfc-2eeb363397f5-trusted-ca\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.561458 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.561302 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d894be86-2b42-4354-aebe-c3533846e9dd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kfhg4\" (UID: \"d894be86-2b42-4354-aebe-c3533846e9dd\") " pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.562842 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.562496 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05a59322-0f9f-427a-8cfc-2eeb363397f5-ca-trust-extracted\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.562842 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.562522 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05a59322-0f9f-427a-8cfc-2eeb363397f5-registry-certificates\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.563122 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.563102 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d894be86-2b42-4354-aebe-c3533846e9dd-crio-socket\") pod \"insights-runtime-extractor-kfhg4\" (UID: \"d894be86-2b42-4354-aebe-c3533846e9dd\") " pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.563445 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.563426 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d894be86-2b42-4354-aebe-c3533846e9dd-data-volume\") pod \"insights-runtime-extractor-kfhg4\" (UID: \"d894be86-2b42-4354-aebe-c3533846e9dd\") " pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.565883 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.565862 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05a59322-0f9f-427a-8cfc-2eeb363397f5-installation-pull-secrets\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.566535 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.566475 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d894be86-2b42-4354-aebe-c3533846e9dd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kfhg4\" (UID: \"d894be86-2b42-4354-aebe-c3533846e9dd\") " pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.567067 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.567023 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05a59322-0f9f-427a-8cfc-2eeb363397f5-registry-tls\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.568625 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.568593 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/86fac976-acfe-4419-9e2a-0f468bed1f6d-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-nchxs\" (UID: \"86fac976-acfe-4419-9e2a-0f468bed1f6d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-nchxs" Jun 03 15:59:23.568721 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.568596 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05a59322-0f9f-427a-8cfc-2eeb363397f5-image-registry-private-configuration\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.572424 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.572305 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05a59322-0f9f-427a-8cfc-2eeb363397f5-bound-sa-token\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.573096 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.573048 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-674pk\" (UniqueName: \"kubernetes.io/projected/d894be86-2b42-4354-aebe-c3533846e9dd-kube-api-access-674pk\") pod \"insights-runtime-extractor-kfhg4\" (UID: \"d894be86-2b42-4354-aebe-c3533846e9dd\") " pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.575662 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.575634 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g5dx\" (UniqueName: \"kubernetes.io/projected/05a59322-0f9f-427a-8cfc-2eeb363397f5-kube-api-access-9g5dx\") pod \"image-registry-658b55885b-dt9vz\" (UID: \"05a59322-0f9f-427a-8cfc-2eeb363397f5\") " pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.603410 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.603381 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-nchxs" Jun 03 15:59:23.610261 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.610230 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kfhg4" Jun 03 15:59:23.619442 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.619414 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:23.636453 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.636425 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-k7xdg"] Jun 03 15:59:23.638756 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:59:23.638720 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f6ff88f_c358_44b6_aa8e_ee0b2f094036.slice/crio-e1e2dd8a21b8f39579750db49938d881ed88c0f05d5e6fca4206775bcc6259ab WatchSource:0}: Error finding container e1e2dd8a21b8f39579750db49938d881ed88c0f05d5e6fca4206775bcc6259ab: Status 404 returned error can't find the container with id e1e2dd8a21b8f39579750db49938d881ed88c0f05d5e6fca4206775bcc6259ab Jun 03 15:59:23.756360 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.756316 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-nchxs"] Jun 03 15:59:23.759587 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:59:23.759555 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86fac976_acfe_4419_9e2a_0f468bed1f6d.slice/crio-48390a54d580acd1e4922f1e60dd717da19e8c52468c2849cdd9960c2266522a WatchSource:0}: Error finding container 48390a54d580acd1e4922f1e60dd717da19e8c52468c2849cdd9960c2266522a: Status 404 returned error can't find the container with id 48390a54d580acd1e4922f1e60dd717da19e8c52468c2849cdd9960c2266522a Jun 03 15:59:23.771980 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.771954 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kfhg4"] Jun 03 15:59:23.775101 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:59:23.775073 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd894be86_2b42_4354_aebe_c3533846e9dd.slice/crio-91f8cd2481a7b36941b2d6a206af2a29497cb5595e8d8bc851e9a1960e999b32 WatchSource:0}: Error finding container 91f8cd2481a7b36941b2d6a206af2a29497cb5595e8d8bc851e9a1960e999b32: Status 404 returned error can't find the container with id 91f8cd2481a7b36941b2d6a206af2a29497cb5595e8d8bc851e9a1960e999b32 Jun 03 15:59:23.791288 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.791241 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-658b55885b-dt9vz"] Jun 03 15:59:23.807888 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:59:23.807847 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05a59322_0f9f_427a_8cfc_2eeb363397f5.slice/crio-6274cd97a2ab89ed9eeb27923a9845279a33af4482c39169dec90e0f65b68e2f WatchSource:0}: Error finding container 6274cd97a2ab89ed9eeb27923a9845279a33af4482c39169dec90e0f65b68e2f: Status 404 returned error can't find the container with id 6274cd97a2ab89ed9eeb27923a9845279a33af4482c39169dec90e0f65b68e2f Jun 03 15:59:23.833628 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.833594 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-nchxs" event={"ID":"86fac976-acfe-4419-9e2a-0f468bed1f6d","Type":"ContainerStarted","Data":"48390a54d580acd1e4922f1e60dd717da19e8c52468c2849cdd9960c2266522a"} Jun 03 15:59:23.834619 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.834594 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k7xdg" event={"ID":"4f6ff88f-c358-44b6-aa8e-ee0b2f094036","Type":"ContainerStarted","Data":"e1e2dd8a21b8f39579750db49938d881ed88c0f05d5e6fca4206775bcc6259ab"} Jun 03 15:59:23.835560 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.835528 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" event={"ID":"05a59322-0f9f-427a-8cfc-2eeb363397f5","Type":"ContainerStarted","Data":"6274cd97a2ab89ed9eeb27923a9845279a33af4482c39169dec90e0f65b68e2f"} Jun 03 15:59:23.836534 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:23.836516 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kfhg4" event={"ID":"d894be86-2b42-4354-aebe-c3533846e9dd","Type":"ContainerStarted","Data":"91f8cd2481a7b36941b2d6a206af2a29497cb5595e8d8bc851e9a1960e999b32"} Jun 03 15:59:24.841279 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:24.841238 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" event={"ID":"05a59322-0f9f-427a-8cfc-2eeb363397f5","Type":"ContainerStarted","Data":"bdc3d78a3e68c3ffd55dda2f72a757ee0fd58a8820c72f30ccd6a0309cf638de"} Jun 03 15:59:24.841758 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:24.841356 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 15:59:24.842805 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:24.842762 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kfhg4" event={"ID":"d894be86-2b42-4354-aebe-c3533846e9dd","Type":"ContainerStarted","Data":"a18cdc8c919bc29a2429c43795592246fa002540fdaca6cbd1eefff0bdf7d1a1"} Jun 03 15:59:24.860042 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:24.859981 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" podStartSLOduration=1.859964299 podStartE2EDuration="1.859964299s" podCreationTimestamp="2026-06-03 15:59:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 15:59:24.859339107 +0000 UTC m=+138.036308919" watchObservedRunningTime="2026-06-03 15:59:24.859964299 +0000 UTC m=+138.036934107" Jun 03 15:59:25.847851 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:25.847805 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kfhg4" event={"ID":"d894be86-2b42-4354-aebe-c3533846e9dd","Type":"ContainerStarted","Data":"c9760b9b065c780bdceb7f01eecfd7b0b771a2665103fa683506372d4d360d46"} Jun 03 15:59:25.849412 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:25.849381 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-nchxs" event={"ID":"86fac976-acfe-4419-9e2a-0f468bed1f6d","Type":"ContainerStarted","Data":"88f1609730adb5624f5c00ecd6dcb6645ce2728bd4b625d14158d9d80df72965"} Jun 03 15:59:25.849657 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:25.849624 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-nchxs" Jun 03 15:59:25.851134 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:25.850977 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k7xdg" event={"ID":"4f6ff88f-c358-44b6-aa8e-ee0b2f094036","Type":"ContainerStarted","Data":"f2b0bd8436e20110831f5600e37cec758b4dfde38178dac841b0f8f6f06d048c"} Jun 03 15:59:25.855246 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:25.855223 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-nchxs" Jun 03 15:59:25.865401 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:25.865358 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-nchxs" podStartSLOduration=1.570710174 podStartE2EDuration="2.865346131s" podCreationTimestamp="2026-06-03 15:59:23 +0000 UTC" firstStartedPulling="2026-06-03 15:59:23.76187987 +0000 UTC m=+136.938849662" lastFinishedPulling="2026-06-03 15:59:25.056515827 +0000 UTC m=+138.233485619" observedRunningTime="2026-06-03 15:59:25.863586622 +0000 UTC m=+139.040556431" watchObservedRunningTime="2026-06-03 15:59:25.865346131 +0000 UTC m=+139.042315940" Jun 03 15:59:25.894429 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:25.894375 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k7xdg" podStartSLOduration=1.4807838709999999 podStartE2EDuration="2.894361189s" podCreationTimestamp="2026-06-03 15:59:23 +0000 UTC" firstStartedPulling="2026-06-03 15:59:23.640987964 +0000 UTC m=+136.817957751" lastFinishedPulling="2026-06-03 15:59:25.054565268 +0000 UTC m=+138.231535069" observedRunningTime="2026-06-03 15:59:25.893396382 +0000 UTC m=+139.070366191" watchObservedRunningTime="2026-06-03 15:59:25.894361189 +0000 UTC m=+139.071330998" Jun 03 15:59:26.855546 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:26.855508 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kfhg4" event={"ID":"d894be86-2b42-4354-aebe-c3533846e9dd","Type":"ContainerStarted","Data":"078c6e15e68ebdd611d853bacea5b5ebb81034857b7b93f2528a25f382d8d2e0"} Jun 03 15:59:26.873189 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:26.873132 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kfhg4" podStartSLOduration=1.3181085129999999 podStartE2EDuration="3.873117566s" podCreationTimestamp="2026-06-03 15:59:23 +0000 UTC" firstStartedPulling="2026-06-03 15:59:23.845731733 +0000 UTC m=+137.022701523" lastFinishedPulling="2026-06-03 15:59:26.400740786 +0000 UTC m=+139.577710576" observedRunningTime="2026-06-03 15:59:26.871244823 +0000 UTC m=+140.048214632" watchObservedRunningTime="2026-06-03 15:59:26.873117566 +0000 UTC m=+140.050087374" Jun 03 15:59:31.666628 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.666593 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-66l9q"] Jun 03 15:59:31.677668 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.677637 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.680681 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.680654 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 03 15:59:31.680909 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.680886 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 03 15:59:31.681264 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.681246 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 03 15:59:31.681365 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.681354 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 03 15:59:31.681456 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.681300 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 03 15:59:31.681550 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.681319 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zj7kz\"" Jun 03 15:59:31.681620 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.681295 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 03 15:59:31.827106 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.827051 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-metrics-client-ca\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.827264 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.827116 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-accelerators-collector-config\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.827264 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.827200 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-sys\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.827374 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.827266 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-root\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.827374 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.827298 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-textfile\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.827374 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.827332 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-wtmp\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.827473 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.827438 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.827514 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.827476 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-tls\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.827514 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.827503 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm8rf\" (UniqueName: \"kubernetes.io/projected/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-kube-api-access-lm8rf\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.928376 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.928282 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-tls\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.928376 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.928337 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lm8rf\" (UniqueName: \"kubernetes.io/projected/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-kube-api-access-lm8rf\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.928376 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.928366 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-metrics-client-ca\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.928640 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.928397 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-accelerators-collector-config\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.928640 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.928429 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-sys\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.928640 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.928508 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-sys\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.928640 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.928618 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-root\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.928853 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.928646 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-textfile\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.928853 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.928692 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-wtmp\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.928853 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.928809 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.928853 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.928814 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-root\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.929110 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.929081 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-accelerators-collector-config\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.929245 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.929117 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-metrics-client-ca\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.929245 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:59:31.929149 2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 03 15:59:31.929245 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.929164 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-textfile\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.929245 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.929204 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-wtmp\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.929245 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:59:31.929209 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-tls podName:1b481d2d-e16a-4f9c-a022-cbcc232ced7b nodeName:}" failed. No retries permitted until 2026-06-03 15:59:32.429190595 +0000 UTC m=+145.606160392 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-tls") pod "node-exporter-66l9q" (UID: "1b481d2d-e16a-4f9c-a022-cbcc232ced7b") : secret "node-exporter-tls" not found Jun 03 15:59:31.931237 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.931214 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:31.935650 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:31.935621 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm8rf\" (UniqueName: \"kubernetes.io/projected/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-kube-api-access-lm8rf\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:32.432744 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:32.432705 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-tls\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:32.435086 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:32.435065 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b481d2d-e16a-4f9c-a022-cbcc232ced7b-node-exporter-tls\") pod \"node-exporter-66l9q\" (UID: \"1b481d2d-e16a-4f9c-a022-cbcc232ced7b\") " pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:32.589175 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:32.589135 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-66l9q" Jun 03 15:59:32.597339 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:59:32.597309 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b481d2d_e16a_4f9c_a022_cbcc232ced7b.slice/crio-f4c95924e3b5407a8562b5938f162724e414e5b34fd9688baf14686f16b2c327 WatchSource:0}: Error finding container f4c95924e3b5407a8562b5938f162724e414e5b34fd9688baf14686f16b2c327: Status 404 returned error can't find the container with id f4c95924e3b5407a8562b5938f162724e414e5b34fd9688baf14686f16b2c327 Jun 03 15:59:32.870895 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:32.870848 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66l9q" event={"ID":"1b481d2d-e16a-4f9c-a022-cbcc232ced7b","Type":"ContainerStarted","Data":"f4c95924e3b5407a8562b5938f162724e414e5b34fd9688baf14686f16b2c327"} Jun 03 15:59:33.874919 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:33.874883 2576 generic.go:358] "Generic (PLEG): container finished" podID="1b481d2d-e16a-4f9c-a022-cbcc232ced7b" containerID="1c77287231a1833b3000d8c7b08e906b935422edd5651f6f98e7cc01ff684ea9" exitCode=0 Jun 03 15:59:33.875292 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:33.874968 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66l9q" event={"ID":"1b481d2d-e16a-4f9c-a022-cbcc232ced7b","Type":"ContainerDied","Data":"1c77287231a1833b3000d8c7b08e906b935422edd5651f6f98e7cc01ff684ea9"} Jun 03 15:59:34.882687 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:34.882650 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66l9q" event={"ID":"1b481d2d-e16a-4f9c-a022-cbcc232ced7b","Type":"ContainerStarted","Data":"19f0adcb781ad47895268c9f54a70f61dd8362c2beb5181050137a6846c93e08"} Jun 03 15:59:34.882687 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:34.882686 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66l9q" event={"ID":"1b481d2d-e16a-4f9c-a022-cbcc232ced7b","Type":"ContainerStarted","Data":"f7b25c048b9870185e059dde4c7bbfc3529d6bc61e53b5b30ba8a4a15cb820a3"} Jun 03 15:59:34.902087 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:34.902027 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-66l9q" podStartSLOduration=2.865458721 podStartE2EDuration="3.902012779s" podCreationTimestamp="2026-06-03 15:59:31 +0000 UTC" firstStartedPulling="2026-06-03 15:59:32.598975498 +0000 UTC m=+145.775945288" lastFinishedPulling="2026-06-03 15:59:33.635529556 +0000 UTC m=+146.812499346" observedRunningTime="2026-06-03 15:59:34.900290151 +0000 UTC m=+148.077259960" watchObservedRunningTime="2026-06-03 15:59:34.902012779 +0000 UTC m=+148.078982587" Jun 03 15:59:36.861052 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.861007 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-555766866-dzx92"] Jun 03 15:59:36.863734 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.863718 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:36.866651 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.866603 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 03 15:59:36.866795 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.866656 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 03 15:59:36.866795 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.866653 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 03 15:59:36.866795 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.866607 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 03 15:59:36.866795 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.866659 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-x7zgx\"" Jun 03 15:59:36.867090 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.867048 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 03 15:59:36.873298 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.873270 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 03 15:59:36.873947 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.873929 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-555766866-dzx92"] Jun 03 15:59:36.970481 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.970440 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d37840e1-e034-43a9-907d-29543d05e389-telemeter-client-tls\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:36.970663 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.970503 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d37840e1-e034-43a9-907d-29543d05e389-serving-certs-ca-bundle\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:36.970663 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.970551 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d37840e1-e034-43a9-907d-29543d05e389-telemeter-trusted-ca-bundle\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:36.970663 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.970588 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d37840e1-e034-43a9-907d-29543d05e389-secret-telemeter-client\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:36.970663 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.970615 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d37840e1-e034-43a9-907d-29543d05e389-metrics-client-ca\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:36.970663 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.970632 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d37840e1-e034-43a9-907d-29543d05e389-federate-client-tls\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:36.970893 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.970660 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d37840e1-e034-43a9-907d-29543d05e389-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:36.970893 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:36.970718 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt4gh\" (UniqueName: \"kubernetes.io/projected/d37840e1-e034-43a9-907d-29543d05e389-kube-api-access-pt4gh\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.071904 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.071871 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d37840e1-e034-43a9-907d-29543d05e389-telemeter-trusted-ca-bundle\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.072044 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.071917 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d37840e1-e034-43a9-907d-29543d05e389-secret-telemeter-client\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.072044 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.071947 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d37840e1-e034-43a9-907d-29543d05e389-metrics-client-ca\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.072044 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.071967 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d37840e1-e034-43a9-907d-29543d05e389-federate-client-tls\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.072186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.072090 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d37840e1-e034-43a9-907d-29543d05e389-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.072186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.072146 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pt4gh\" (UniqueName: \"kubernetes.io/projected/d37840e1-e034-43a9-907d-29543d05e389-kube-api-access-pt4gh\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.072186 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.072181 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d37840e1-e034-43a9-907d-29543d05e389-telemeter-client-tls\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.072334 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.072253 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d37840e1-e034-43a9-907d-29543d05e389-serving-certs-ca-bundle\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.072660 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.072625 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d37840e1-e034-43a9-907d-29543d05e389-metrics-client-ca\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.072905 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.072882 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d37840e1-e034-43a9-907d-29543d05e389-telemeter-trusted-ca-bundle\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.073018 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.072888 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d37840e1-e034-43a9-907d-29543d05e389-serving-certs-ca-bundle\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.074562 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.074542 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d37840e1-e034-43a9-907d-29543d05e389-secret-telemeter-client\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.074735 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.074687 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d37840e1-e034-43a9-907d-29543d05e389-federate-client-tls\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.075003 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.074982 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d37840e1-e034-43a9-907d-29543d05e389-telemeter-client-tls\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.075080 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.075016 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d37840e1-e034-43a9-907d-29543d05e389-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.080367 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.080343 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt4gh\" (UniqueName: \"kubernetes.io/projected/d37840e1-e034-43a9-907d-29543d05e389-kube-api-access-pt4gh\") pod \"telemeter-client-555766866-dzx92\" (UID: \"d37840e1-e034-43a9-907d-29543d05e389\") " pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.174830 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.174744 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-555766866-dzx92" Jun 03 15:59:37.302977 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.302952 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-555766866-dzx92"] Jun 03 15:59:37.305360 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:59:37.305329 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd37840e1_e034_43a9_907d_29543d05e389.slice/crio-20862a72581740f511ec66d6370ba813f062989f8e4bcf80e48dc7926204ffb4 WatchSource:0}: Error finding container 20862a72581740f511ec66d6370ba813f062989f8e4bcf80e48dc7926204ffb4: Status 404 returned error can't find the container with id 20862a72581740f511ec66d6370ba813f062989f8e4bcf80e48dc7926204ffb4 Jun 03 15:59:37.865617 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.865580 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 15:59:37.868682 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.868662 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.875478 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.875452 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 03 15:59:37.875478 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.875468 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 03 15:59:37.875657 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.875452 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 03 15:59:37.876362 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.876347 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7nhh69g2qvo0o\"" Jun 03 15:59:37.876461 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.876447 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 03 15:59:37.877474 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.877456 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 03 15:59:37.877586 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.877477 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bskkp\"" Jun 03 15:59:37.877586 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.877533 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 03 15:59:37.877736 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.877722 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 03 15:59:37.878373 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.878360 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 03 15:59:37.880096 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.880079 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 03 15:59:37.880303 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.880290 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 03 15:59:37.881682 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.881669 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 03 15:59:37.885977 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.885959 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 03 15:59:37.891988 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.891966 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-555766866-dzx92" event={"ID":"d37840e1-e034-43a9-907d-29543d05e389","Type":"ContainerStarted","Data":"20862a72581740f511ec66d6370ba813f062989f8e4bcf80e48dc7926204ffb4"} Jun 03 15:59:37.892342 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.892326 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 03 15:59:37.908634 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.908612 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 15:59:37.980585 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.980548 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-config\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.980786 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.980602 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pb98\" (UniqueName: \"kubernetes.io/projected/ae32d7e7-853b-4459-b854-aa33aeacc8f9-kube-api-access-7pb98\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.980786 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.980630 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.980786 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.980665 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.980920 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.980760 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.980920 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.980823 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.980920 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.980876 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae32d7e7-853b-4459-b854-aa33aeacc8f9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.980920 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.980914 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.981110 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.980938 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.981110 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.980955 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.981110 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.981001 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.981110 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.981049 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.981110 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.981081 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.981306 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.981114 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-web-config\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.981306 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.981164 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.981306 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.981205 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.981306 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.981249 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae32d7e7-853b-4459-b854-aa33aeacc8f9-config-out\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:37.981306 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:37.981293 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082117 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082074 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082117 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082115 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082384 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082133 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082384 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082153 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082384 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082175 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082384 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082209 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-web-config\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082572 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082398 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082572 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082464 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082572 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082496 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae32d7e7-853b-4459-b854-aa33aeacc8f9-config-out\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082572 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082521 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082572 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082566 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-config\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082859 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082591 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7pb98\" (UniqueName: \"kubernetes.io/projected/ae32d7e7-853b-4459-b854-aa33aeacc8f9-kube-api-access-7pb98\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082859 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082629 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082859 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082674 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082859 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082716 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082859 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082749 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082859 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082809 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae32d7e7-853b-4459-b854-aa33aeacc8f9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.082859 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.082841 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.083214 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.083047 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.083324 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.083298 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.083964 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.083937 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.086210 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.086186 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.086416 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.086349 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.086565 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.086542 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.086628 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.086560 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.086833 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.086740 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.086833 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.086746 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-web-config\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.087004 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.086917 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.087263 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.087236 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.087787 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.087739 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.087998 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.087976 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae32d7e7-853b-4459-b854-aa33aeacc8f9-config-out\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.088496 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.088466 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae32d7e7-853b-4459-b854-aa33aeacc8f9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.088567 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.088498 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.088859 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.088838 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.089652 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.089637 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-config\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.093968 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.093947 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pb98\" (UniqueName: \"kubernetes.io/projected/ae32d7e7-853b-4459-b854-aa33aeacc8f9-kube-api-access-7pb98\") pod \"prometheus-k8s-0\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.178140 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.178039 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:38.314900 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.314876 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 15:59:38.317257 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:59:38.317225 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae32d7e7_853b_4459_b854_aa33aeacc8f9.slice/crio-3d2d6ca1cd329bfb01bcfc8ff9a939fa3654a7ad7455578f7dfd0db6d1715d52 WatchSource:0}: Error finding container 3d2d6ca1cd329bfb01bcfc8ff9a939fa3654a7ad7455578f7dfd0db6d1715d52: Status 404 returned error can't find the container with id 3d2d6ca1cd329bfb01bcfc8ff9a939fa3654a7ad7455578f7dfd0db6d1715d52 Jun 03 15:59:38.896966 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:38.896926 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae32d7e7-853b-4459-b854-aa33aeacc8f9","Type":"ContainerStarted","Data":"3d2d6ca1cd329bfb01bcfc8ff9a939fa3654a7ad7455578f7dfd0db6d1715d52"} Jun 03 15:59:39.900663 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:39.900616 2576 generic.go:358] "Generic (PLEG): container finished" podID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerID="9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df" exitCode=0 Jun 03 15:59:39.901147 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:39.900697 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae32d7e7-853b-4459-b854-aa33aeacc8f9","Type":"ContainerDied","Data":"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df"} Jun 03 15:59:39.902737 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:39.902708 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-555766866-dzx92" event={"ID":"d37840e1-e034-43a9-907d-29543d05e389","Type":"ContainerStarted","Data":"91765ff66b62a9f59aff3abdd0036cd247345f55d70edebdbc9b0011a5ca2cd7"} Jun 03 15:59:39.902849 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:39.902748 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-555766866-dzx92" event={"ID":"d37840e1-e034-43a9-907d-29543d05e389","Type":"ContainerStarted","Data":"f6484dff50b7b5ba6e136a96331babc800d415a6b38f221e5869c411b8039b36"} Jun 03 15:59:39.902849 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:39.902763 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-555766866-dzx92" event={"ID":"d37840e1-e034-43a9-907d-29543d05e389","Type":"ContainerStarted","Data":"79ba6fff05e36eafde8de1586a4988d89de4b2cc3d16092e2e273308065d8eaf"} Jun 03 15:59:39.948730 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:39.948676 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-555766866-dzx92" podStartSLOduration=1.616902904 podStartE2EDuration="3.948662458s" podCreationTimestamp="2026-06-03 15:59:36 +0000 UTC" firstStartedPulling="2026-06-03 15:59:37.307209597 +0000 UTC m=+150.484179387" lastFinishedPulling="2026-06-03 15:59:39.63896915 +0000 UTC m=+152.815938941" observedRunningTime="2026-06-03 15:59:39.946493256 +0000 UTC m=+153.123463064" watchObservedRunningTime="2026-06-03 15:59:39.948662458 +0000 UTC m=+153.125632278" Jun 03 15:59:42.915547 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:42.915512 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae32d7e7-853b-4459-b854-aa33aeacc8f9","Type":"ContainerStarted","Data":"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c"} Jun 03 15:59:43.625359 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:43.625314 2576 patch_prober.go:28] interesting pod/image-registry-658b55885b-dt9vz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 03 15:59:43.625525 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:43.625377 2576 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" podUID="05a59322-0f9f-427a-8cfc-2eeb363397f5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 15:59:43.760456 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:59:43.760405 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-76kk6" podUID="649647f7-81ce-4ed1-80d4-7ab2cabda43d" Jun 03 15:59:43.783085 ip-10-0-128-61 kubenswrapper[2576]: E0603 15:59:43.783035 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-l2jxd" podUID="1e3e630c-6304-40da-935c-2dc17eb2ddd0" Jun 03 15:59:43.921062 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:43.920969 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae32d7e7-853b-4459-b854-aa33aeacc8f9","Type":"ContainerStarted","Data":"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae"} Jun 03 15:59:43.921062 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:43.920991 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:59:43.921062 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:43.921022 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-76kk6" Jun 03 15:59:44.926543 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:44.926514 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae32d7e7-853b-4459-b854-aa33aeacc8f9","Type":"ContainerStarted","Data":"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22"} Jun 03 15:59:44.926875 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:44.926550 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae32d7e7-853b-4459-b854-aa33aeacc8f9","Type":"ContainerStarted","Data":"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b"} Jun 03 15:59:45.855582 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:45.855548 2576 patch_prober.go:28] interesting pod/image-registry-658b55885b-dt9vz container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 03 15:59:45.855756 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:45.855614 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" podUID="05a59322-0f9f-427a-8cfc-2eeb363397f5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 15:59:45.931928 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:45.931891 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae32d7e7-853b-4459-b854-aa33aeacc8f9","Type":"ContainerStarted","Data":"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0"} Jun 03 15:59:45.931928 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:45.931930 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae32d7e7-853b-4459-b854-aa33aeacc8f9","Type":"ContainerStarted","Data":"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319"} Jun 03 15:59:45.960168 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:45.960107 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.535476403 podStartE2EDuration="8.960087962s" podCreationTimestamp="2026-06-03 15:59:37 +0000 UTC" firstStartedPulling="2026-06-03 15:59:38.319055964 +0000 UTC m=+151.496025754" lastFinishedPulling="2026-06-03 15:59:44.743667522 +0000 UTC m=+157.920637313" observedRunningTime="2026-06-03 15:59:45.957627893 +0000 UTC m=+159.134597701" watchObservedRunningTime="2026-06-03 15:59:45.960087962 +0000 UTC m=+159.137057771" Jun 03 15:59:48.178878 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:48.178830 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 15:59:48.581215 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:48.581167 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:59:48.583499 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:48.583465 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649647f7-81ce-4ed1-80d4-7ab2cabda43d-metrics-tls\") pod \"dns-default-76kk6\" (UID: \"649647f7-81ce-4ed1-80d4-7ab2cabda43d\") " pod="openshift-dns/dns-default-76kk6" Jun 03 15:59:48.681910 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:48.681864 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert\") pod \"ingress-canary-l2jxd\" (UID: \"1e3e630c-6304-40da-935c-2dc17eb2ddd0\") " pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:59:48.684273 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:48.684251 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e3e630c-6304-40da-935c-2dc17eb2ddd0-cert\") pod \"ingress-canary-l2jxd\" (UID: \"1e3e630c-6304-40da-935c-2dc17eb2ddd0\") " pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:59:48.724717 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:48.724678 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-drwlt\"" Jun 03 15:59:48.725557 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:48.725539 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vbxw8\"" Jun 03 15:59:48.733314 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:48.733290 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-76kk6" Jun 03 15:59:48.733417 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:48.733353 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l2jxd" Jun 03 15:59:48.875206 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:48.875057 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-76kk6"] Jun 03 15:59:48.878095 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:59:48.878059 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod649647f7_81ce_4ed1_80d4_7ab2cabda43d.slice/crio-b4580525cf27aa4944ff12fb06a5c9741c7108536c85473382b020c2b34f6eef WatchSource:0}: Error finding container b4580525cf27aa4944ff12fb06a5c9741c7108536c85473382b020c2b34f6eef: Status 404 returned error can't find the container with id b4580525cf27aa4944ff12fb06a5c9741c7108536c85473382b020c2b34f6eef Jun 03 15:59:48.890889 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:48.890849 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l2jxd"] Jun 03 15:59:48.893695 ip-10-0-128-61 kubenswrapper[2576]: W0603 15:59:48.893671 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3e630c_6304_40da_935c_2dc17eb2ddd0.slice/crio-bdecfe93b06f378fcafb8a6818f2fe41620e7c8374f5a696d656cac4db120a09 WatchSource:0}: Error finding container bdecfe93b06f378fcafb8a6818f2fe41620e7c8374f5a696d656cac4db120a09: Status 404 returned error can't find the container with id bdecfe93b06f378fcafb8a6818f2fe41620e7c8374f5a696d656cac4db120a09 Jun 03 15:59:48.941075 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:48.941037 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l2jxd" event={"ID":"1e3e630c-6304-40da-935c-2dc17eb2ddd0","Type":"ContainerStarted","Data":"bdecfe93b06f378fcafb8a6818f2fe41620e7c8374f5a696d656cac4db120a09"} Jun 03 15:59:48.942141 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:48.942114 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-76kk6" event={"ID":"649647f7-81ce-4ed1-80d4-7ab2cabda43d","Type":"ContainerStarted","Data":"b4580525cf27aa4944ff12fb06a5c9741c7108536c85473382b020c2b34f6eef"} Jun 03 15:59:50.951179 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:50.951141 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-76kk6" event={"ID":"649647f7-81ce-4ed1-80d4-7ab2cabda43d","Type":"ContainerStarted","Data":"4f8abc518cff9f30bf1940d38ce1c174a723a7d256fbde47dac7a064ecc31a08"} Jun 03 15:59:50.951179 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:50.951176 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-76kk6" event={"ID":"649647f7-81ce-4ed1-80d4-7ab2cabda43d","Type":"ContainerStarted","Data":"ac2e76a387db252be28524197959a5315e94fa04da3767d48941d3a575199326"} Jun 03 15:59:50.951675 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:50.951194 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-76kk6" Jun 03 15:59:50.952448 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:50.952428 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l2jxd" event={"ID":"1e3e630c-6304-40da-935c-2dc17eb2ddd0","Type":"ContainerStarted","Data":"b01c1dacd7cc1a5ba10d4737b283b54d9ede6b2c969d59747c802f94236312d8"} Jun 03 15:59:50.968190 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:50.968142 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-76kk6" podStartSLOduration=129.250791817 podStartE2EDuration="2m10.968127813s" podCreationTimestamp="2026-06-03 15:57:40 +0000 UTC" firstStartedPulling="2026-06-03 15:59:48.880008943 +0000 UTC m=+162.056978730" lastFinishedPulling="2026-06-03 15:59:50.59734494 +0000 UTC m=+163.774314726" observedRunningTime="2026-06-03 15:59:50.966788729 +0000 UTC m=+164.143758551" watchObservedRunningTime="2026-06-03 15:59:50.968127813 +0000 UTC m=+164.145097621" Jun 03 15:59:50.982233 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:50.982186 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-l2jxd" podStartSLOduration=129.278399884 podStartE2EDuration="2m10.982170802s" podCreationTimestamp="2026-06-03 15:57:40 +0000 UTC" firstStartedPulling="2026-06-03 15:59:48.89584288 +0000 UTC m=+162.072812667" lastFinishedPulling="2026-06-03 15:59:50.599613795 +0000 UTC m=+163.776583585" observedRunningTime="2026-06-03 15:59:50.981330083 +0000 UTC m=+164.158299897" watchObservedRunningTime="2026-06-03 15:59:50.982170802 +0000 UTC m=+164.159140658" Jun 03 15:59:53.623956 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:53.623921 2576 patch_prober.go:28] interesting pod/image-registry-658b55885b-dt9vz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 03 15:59:53.624397 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:53.623987 2576 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" podUID="05a59322-0f9f-427a-8cfc-2eeb363397f5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 15:59:55.855381 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:55.855347 2576 patch_prober.go:28] interesting pod/image-registry-658b55885b-dt9vz container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 03 15:59:55.855757 ip-10-0-128-61 kubenswrapper[2576]: I0603 15:59:55.855400 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" podUID="05a59322-0f9f-427a-8cfc-2eeb363397f5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 16:00:00.957450 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:00.957415 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-76kk6" Jun 03 16:00:03.624278 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:03.624245 2576 patch_prober.go:28] interesting pod/image-registry-658b55885b-dt9vz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 03 16:00:03.624655 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:03.624301 2576 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" podUID="05a59322-0f9f-427a-8cfc-2eeb363397f5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 16:00:03.624655 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:03.624342 2576 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 16:00:03.624855 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:03.624814 2576 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"bdc3d78a3e68c3ffd55dda2f72a757ee0fd58a8820c72f30ccd6a0309cf638de"} pod="openshift-image-registry/image-registry-658b55885b-dt9vz" containerMessage="Container registry failed liveness probe, will be restarted" Jun 03 16:00:03.628606 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:03.628574 2576 patch_prober.go:28] interesting pod/image-registry-658b55885b-dt9vz container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 03 16:00:03.628746 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:03.628642 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" podUID="05a59322-0f9f-427a-8cfc-2eeb363397f5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 16:00:10.012252 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:10.012220 2576 generic.go:358] "Generic (PLEG): container finished" podID="ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad" containerID="045d2fbd9ef9b3da944a512d88778b5e0062ededc341784022a8406ad57b94f5" exitCode=255 Jun 03 16:00:10.012665 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:10.012296 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d8699c84-md6kn" event={"ID":"ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad","Type":"ContainerDied","Data":"045d2fbd9ef9b3da944a512d88778b5e0062ededc341784022a8406ad57b94f5"} Jun 03 16:00:10.018252 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:10.018231 2576 scope.go:117] "RemoveContainer" containerID="045d2fbd9ef9b3da944a512d88778b5e0062ededc341784022a8406ad57b94f5" Jun 03 16:00:11.017022 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:11.016984 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d8699c84-md6kn" event={"ID":"ef5f3330-ceb0-4224-86ee-0bcfd2cee9ad","Type":"ContainerStarted","Data":"2b08d1f35ed4e435c7b94c39e6999a014c36d41a1b52976fedfe4cfef5bacb37"} Jun 03 16:00:13.629752 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:13.629724 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 16:00:16.032284 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:16.032247 2576 generic.go:358] "Generic (PLEG): container finished" podID="6ec5c06b-c6ab-4826-9076-93046a57fb73" containerID="404b728fcd1d2c95a568d2a6462bce12f18babf334b0a5120250daaa6e75efae" exitCode=0 Jun 03 16:00:16.032648 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:16.032322 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv" event={"ID":"6ec5c06b-c6ab-4826-9076-93046a57fb73","Type":"ContainerDied","Data":"404b728fcd1d2c95a568d2a6462bce12f18babf334b0a5120250daaa6e75efae"} Jun 03 16:00:16.032690 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:16.032652 2576 scope.go:117] "RemoveContainer" containerID="404b728fcd1d2c95a568d2a6462bce12f18babf334b0a5120250daaa6e75efae" Jun 03 16:00:17.037688 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:17.037656 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-vbrqv" event={"ID":"6ec5c06b-c6ab-4826-9076-93046a57fb73","Type":"ContainerStarted","Data":"00ff4acbf021980e870f11dac44944db73e6d943f607b03f252dd789551a8f38"} Jun 03 16:00:28.646673 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:28.646627 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" podUID="05a59322-0f9f-427a-8cfc-2eeb363397f5" containerName="registry" containerID="cri-o://bdc3d78a3e68c3ffd55dda2f72a757ee0fd58a8820c72f30ccd6a0309cf638de" gracePeriod=30 Jun 03 16:00:30.079264 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:30.079156 2576 generic.go:358] "Generic (PLEG): container finished" podID="05a59322-0f9f-427a-8cfc-2eeb363397f5" containerID="bdc3d78a3e68c3ffd55dda2f72a757ee0fd58a8820c72f30ccd6a0309cf638de" exitCode=0 Jun 03 16:00:30.079720 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:30.079316 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" event={"ID":"05a59322-0f9f-427a-8cfc-2eeb363397f5","Type":"ContainerDied","Data":"bdc3d78a3e68c3ffd55dda2f72a757ee0fd58a8820c72f30ccd6a0309cf638de"} Jun 03 16:00:30.079720 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:30.079351 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" event={"ID":"05a59322-0f9f-427a-8cfc-2eeb363397f5","Type":"ContainerStarted","Data":"2c9b6b1b7431a41bb7944c3b20043652043506fd94f9edf97f4dab8e39af757e"} Jun 03 16:00:30.080444 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:30.080240 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 16:00:38.178860 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:38.178810 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:38.194409 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:38.194381 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:39.119519 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:39.119485 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:52.089159 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:52.089125 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-658b55885b-dt9vz" Jun 03 16:00:56.180297 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.180263 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 16:00:56.180926 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.180732 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="prometheus" containerID="cri-o://9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c" gracePeriod=600 Jun 03 16:00:56.180926 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.180751 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="thanos-sidecar" containerID="cri-o://8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b" gracePeriod=600 Jun 03 16:00:56.180926 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.180740 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="kube-rbac-proxy-web" containerID="cri-o://97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22" gracePeriod=600 Jun 03 16:00:56.180926 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.180781 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="kube-rbac-proxy-thanos" containerID="cri-o://27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0" gracePeriod=600 Jun 03 16:00:56.180926 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.180816 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="config-reloader" containerID="cri-o://3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae" gracePeriod=600 Jun 03 16:00:56.180926 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.180832 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="kube-rbac-proxy" containerID="cri-o://52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319" gracePeriod=600 Jun 03 16:00:56.415901 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.415874 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:56.450003 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.449970 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-kube-rbac-proxy\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450203 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450020 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-serving-certs-ca-bundle\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450203 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450050 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-config\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450203 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450072 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-k8s-rulefiles-0\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450203 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450105 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pb98\" (UniqueName: \"kubernetes.io/projected/ae32d7e7-853b-4459-b854-aa33aeacc8f9-kube-api-access-7pb98\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450203 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450129 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-tls\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450203 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450167 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae32d7e7-853b-4459-b854-aa33aeacc8f9-tls-assets\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450203 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450194 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-web-config\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450564 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450244 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-k8s-db\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450564 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450275 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450564 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450305 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-metrics-client-ca\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450564 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450334 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-grpc-tls\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450564 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450379 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-metrics-client-certs\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450564 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450411 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450564 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450443 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-trusted-ca-bundle\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450564 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450471 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae32d7e7-853b-4459-b854-aa33aeacc8f9-config-out\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450564 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450499 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-thanos-prometheus-http-client-file\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.450564 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450533 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-kubelet-serving-ca-bundle\") pod \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\" (UID: \"ae32d7e7-853b-4459-b854-aa33aeacc8f9\") " Jun 03 16:00:56.451094 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450824 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:00:56.451094 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.450970 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:00:56.452928 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.451704 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:00:56.452928 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.452885 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:00:56.453127 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.452940 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:00:56.453258 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.453019 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:00:56.456011 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.455981 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:00:56.456132 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.456005 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:00:56.457713 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.457677 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:00:56.457713 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.457702 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:00:56.457955 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.457887 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:00:56.457955 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.457909 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:00:56.458062 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.458038 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-config" (OuterVolumeSpecName: "config") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:00:56.458121 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.458101 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae32d7e7-853b-4459-b854-aa33aeacc8f9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:00:56.458168 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.458145 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae32d7e7-853b-4459-b854-aa33aeacc8f9-kube-api-access-7pb98" (OuterVolumeSpecName: "kube-api-access-7pb98") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "kube-api-access-7pb98". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:00:56.458291 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.458269 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae32d7e7-853b-4459-b854-aa33aeacc8f9-config-out" (OuterVolumeSpecName: "config-out") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:00:56.458510 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.458492 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:00:56.468277 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.468246 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-web-config" (OuterVolumeSpecName: "web-config") pod "ae32d7e7-853b-4459-b854-aa33aeacc8f9" (UID: "ae32d7e7-853b-4459-b854-aa33aeacc8f9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:00:56.551402 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551364 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-k8s-db\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551402 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551397 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551402 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551410 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-metrics-client-ca\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551632 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551420 2576 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-grpc-tls\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551632 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551431 2576 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-metrics-client-certs\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551632 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551440 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551632 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551450 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551632 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551460 2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae32d7e7-853b-4459-b854-aa33aeacc8f9-config-out\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551632 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551469 2576 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551632 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551478 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551632 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551486 2576 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-kube-rbac-proxy\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551632 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551495 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551632 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551504 2576 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-config\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551632 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551513 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ae32d7e7-853b-4459-b854-aa33aeacc8f9-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551632 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551521 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pb98\" (UniqueName: \"kubernetes.io/projected/ae32d7e7-853b-4459-b854-aa33aeacc8f9-kube-api-access-7pb98\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551632 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551529 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551632 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551539 2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae32d7e7-853b-4459-b854-aa33aeacc8f9-tls-assets\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:56.551632 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:56.551547 2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae32d7e7-853b-4459-b854-aa33aeacc8f9-web-config\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:00:57.158073 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.158034 2576 generic.go:358] "Generic (PLEG): container finished" podID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerID="27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0" exitCode=0 Jun 03 16:00:57.158073 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.158064 2576 generic.go:358] "Generic (PLEG): container finished" podID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerID="52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319" exitCode=0 Jun 03 16:00:57.158073 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.158074 2576 generic.go:358] "Generic (PLEG): container finished" podID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerID="97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22" exitCode=0 Jun 03 16:00:57.158073 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.158081 2576 generic.go:358] "Generic (PLEG): container finished" podID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerID="8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b" exitCode=0 Jun 03 16:00:57.158455 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.158075 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae32d7e7-853b-4459-b854-aa33aeacc8f9","Type":"ContainerDied","Data":"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0"} Jun 03 16:00:57.158455 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.158090 2576 generic.go:358] "Generic (PLEG): container finished" podID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerID="3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae" exitCode=0 Jun 03 16:00:57.158455 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.158099 2576 generic.go:358] "Generic (PLEG): container finished" podID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerID="9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c" exitCode=0 Jun 03 16:00:57.158455 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.158119 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae32d7e7-853b-4459-b854-aa33aeacc8f9","Type":"ContainerDied","Data":"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319"} Jun 03 16:00:57.158455 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.158134 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae32d7e7-853b-4459-b854-aa33aeacc8f9","Type":"ContainerDied","Data":"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22"} Jun 03 16:00:57.158455 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.158147 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae32d7e7-853b-4459-b854-aa33aeacc8f9","Type":"ContainerDied","Data":"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b"} Jun 03 16:00:57.158455 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.158154 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.158455 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.158171 2576 scope.go:117] "RemoveContainer" containerID="27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0" Jun 03 16:00:57.158455 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.158161 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae32d7e7-853b-4459-b854-aa33aeacc8f9","Type":"ContainerDied","Data":"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae"} Jun 03 16:00:57.158455 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.158271 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae32d7e7-853b-4459-b854-aa33aeacc8f9","Type":"ContainerDied","Data":"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c"} Jun 03 16:00:57.158455 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.158299 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae32d7e7-853b-4459-b854-aa33aeacc8f9","Type":"ContainerDied","Data":"3d2d6ca1cd329bfb01bcfc8ff9a939fa3654a7ad7455578f7dfd0db6d1715d52"} Jun 03 16:00:57.166527 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.166496 2576 scope.go:117] "RemoveContainer" containerID="52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319" Jun 03 16:00:57.174293 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.174274 2576 scope.go:117] "RemoveContainer" containerID="97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22" Jun 03 16:00:57.181735 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.181590 2576 scope.go:117] "RemoveContainer" containerID="8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b" Jun 03 16:00:57.182127 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.182105 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 16:00:57.185587 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.185564 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 16:00:57.189104 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.189082 2576 scope.go:117] "RemoveContainer" containerID="3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae" Jun 03 16:00:57.207467 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.207443 2576 scope.go:117] "RemoveContainer" containerID="9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c" Jun 03 16:00:57.215054 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.215029 2576 scope.go:117] "RemoveContainer" containerID="9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df" Jun 03 16:00:57.216447 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216429 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 16:00:57.216745 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216723 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="kube-rbac-proxy-thanos" Jun 03 16:00:57.216745 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216740 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="kube-rbac-proxy-thanos" Jun 03 16:00:57.216852 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216756 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="thanos-sidecar" Jun 03 16:00:57.216852 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216763 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="thanos-sidecar" Jun 03 16:00:57.216852 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216804 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="init-config-reloader" Jun 03 16:00:57.216852 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216814 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="init-config-reloader" Jun 03 16:00:57.216852 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216821 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="kube-rbac-proxy-web" Jun 03 16:00:57.216852 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216826 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="kube-rbac-proxy-web" Jun 03 16:00:57.216852 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216834 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="config-reloader" Jun 03 16:00:57.216852 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216839 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="config-reloader" Jun 03 16:00:57.216852 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216845 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="prometheus" Jun 03 16:00:57.216852 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216849 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="prometheus" Jun 03 16:00:57.217172 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216859 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="kube-rbac-proxy" Jun 03 16:00:57.217172 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216864 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="kube-rbac-proxy" Jun 03 16:00:57.217172 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216914 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="kube-rbac-proxy" Jun 03 16:00:57.217172 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216923 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="thanos-sidecar" Jun 03 16:00:57.217172 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216928 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="kube-rbac-proxy-web" Jun 03 16:00:57.217172 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216936 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="prometheus" Jun 03 16:00:57.217172 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216941 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="config-reloader" Jun 03 16:00:57.217172 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.216947 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" containerName="kube-rbac-proxy-thanos" Jun 03 16:00:57.222107 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.222090 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.222588 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.222569 2576 scope.go:117] "RemoveContainer" containerID="27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0" Jun 03 16:00:57.222895 ip-10-0-128-61 kubenswrapper[2576]: E0603 16:00:57.222874 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0\": container with ID starting with 27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0 not found: ID does not exist" containerID="27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0" Jun 03 16:00:57.222988 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.222907 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0"} err="failed to get container status \"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0\": rpc error: code = NotFound desc = could not find container \"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0\": container with ID starting with 27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0 not found: ID does not exist" Jun 03 16:00:57.222988 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.222934 2576 scope.go:117] "RemoveContainer" containerID="52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319" Jun 03 16:00:57.223223 ip-10-0-128-61 kubenswrapper[2576]: E0603 16:00:57.223199 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319\": container with ID starting with 52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319 not found: ID does not exist" containerID="52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319" Jun 03 16:00:57.223266 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.223232 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319"} err="failed to get container status \"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319\": rpc error: code = NotFound desc = could not find container \"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319\": container with ID starting with 52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319 not found: ID does not exist" Jun 03 16:00:57.223266 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.223255 2576 scope.go:117] "RemoveContainer" containerID="97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22" Jun 03 16:00:57.223479 ip-10-0-128-61 kubenswrapper[2576]: E0603 16:00:57.223460 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22\": container with ID starting with 97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22 not found: ID does not exist" containerID="97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22" Jun 03 16:00:57.223546 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.223487 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22"} err="failed to get container status \"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22\": rpc error: code = NotFound desc = could not find container \"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22\": container with ID starting with 97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22 not found: ID does not exist" Jun 03 16:00:57.223546 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.223510 2576 scope.go:117] "RemoveContainer" containerID="8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b" Jun 03 16:00:57.223734 ip-10-0-128-61 kubenswrapper[2576]: E0603 16:00:57.223720 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b\": container with ID starting with 8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b not found: ID does not exist" containerID="8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b" Jun 03 16:00:57.223858 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.223741 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b"} err="failed to get container status \"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b\": rpc error: code = NotFound desc = could not find container \"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b\": container with ID starting with 8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b not found: ID does not exist" Jun 03 16:00:57.223858 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.223759 2576 scope.go:117] "RemoveContainer" containerID="3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae" Jun 03 16:00:57.224015 ip-10-0-128-61 kubenswrapper[2576]: E0603 16:00:57.223996 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae\": container with ID starting with 3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae not found: ID does not exist" containerID="3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae" Jun 03 16:00:57.224069 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.224023 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae"} err="failed to get container status \"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae\": rpc error: code = NotFound desc = could not find container \"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae\": container with ID starting with 3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae not found: ID does not exist" Jun 03 16:00:57.224069 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.224042 2576 scope.go:117] "RemoveContainer" containerID="9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c" Jun 03 16:00:57.224301 ip-10-0-128-61 kubenswrapper[2576]: E0603 16:00:57.224282 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c\": container with ID starting with 9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c not found: ID does not exist" containerID="9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c" Jun 03 16:00:57.224358 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.224306 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c"} err="failed to get container status \"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c\": rpc error: code = NotFound desc = could not find container \"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c\": container with ID starting with 9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c not found: ID does not exist" Jun 03 16:00:57.224358 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.224325 2576 scope.go:117] "RemoveContainer" containerID="9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df" Jun 03 16:00:57.224572 ip-10-0-128-61 kubenswrapper[2576]: E0603 16:00:57.224544 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df\": container with ID starting with 9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df not found: ID does not exist" containerID="9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df" Jun 03 16:00:57.224658 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.224575 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df"} err="failed to get container status \"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df\": rpc error: code = NotFound desc = could not find container \"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df\": container with ID starting with 9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df not found: ID does not exist" Jun 03 16:00:57.224658 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.224597 2576 scope.go:117] "RemoveContainer" containerID="27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0" Jun 03 16:00:57.224825 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.224738 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 03 16:00:57.224891 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.224837 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 03 16:00:57.224891 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.224841 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 03 16:00:57.224986 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.224889 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0"} err="failed to get container status \"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0\": rpc error: code = NotFound desc = could not find container \"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0\": container with ID starting with 27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0 not found: ID does not exist" Jun 03 16:00:57.224986 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.224909 2576 scope.go:117] "RemoveContainer" containerID="52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319" Jun 03 16:00:57.225083 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.224985 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7nhh69g2qvo0o\"" Jun 03 16:00:57.225083 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.225016 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 03 16:00:57.225083 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.225076 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 03 16:00:57.225208 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.225081 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 03 16:00:57.225208 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.225199 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 03 16:00:57.225291 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.225202 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 03 16:00:57.225343 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.225288 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 03 16:00:57.225343 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.225306 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319"} err="failed to get container status \"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319\": rpc error: code = NotFound desc = could not find container \"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319\": container with ID starting with 52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319 not found: ID does not exist" Jun 03 16:00:57.225343 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.225327 2576 scope.go:117] "RemoveContainer" containerID="97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22" Jun 03 16:00:57.225487 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.225428 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 03 16:00:57.225487 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.225462 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bskkp\"" Jun 03 16:00:57.225578 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.225491 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 03 16:00:57.225662 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.225632 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22"} err="failed to get container status \"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22\": rpc error: code = NotFound desc = could not find container \"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22\": container with ID starting with 97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22 not found: ID does not exist" Jun 03 16:00:57.225723 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.225664 2576 scope.go:117] "RemoveContainer" containerID="8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b" Jun 03 16:00:57.225933 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.225907 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b"} err="failed to get container status \"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b\": rpc error: code = NotFound desc = could not find container \"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b\": container with ID starting with 8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b not found: ID does not exist" Jun 03 16:00:57.226031 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.225935 2576 scope.go:117] "RemoveContainer" containerID="3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae" Jun 03 16:00:57.226217 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.226186 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae"} err="failed to get container status \"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae\": rpc error: code = NotFound desc = could not find container \"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae\": container with ID starting with 3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae not found: ID does not exist" Jun 03 16:00:57.226217 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.226216 2576 scope.go:117] "RemoveContainer" containerID="9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c" Jun 03 16:00:57.226473 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.226437 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c"} err="failed to get container status \"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c\": rpc error: code = NotFound desc = could not find container \"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c\": container with ID starting with 9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c not found: ID does not exist" Jun 03 16:00:57.226473 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.226473 2576 scope.go:117] "RemoveContainer" containerID="9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df" Jun 03 16:00:57.226764 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.226745 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df"} err="failed to get container status \"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df\": rpc error: code = NotFound desc = could not find container \"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df\": container with ID starting with 9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df not found: ID does not exist" Jun 03 16:00:57.226764 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.226763 2576 scope.go:117] "RemoveContainer" containerID="27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0" Jun 03 16:00:57.227064 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.227030 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0"} err="failed to get container status \"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0\": rpc error: code = NotFound desc = could not find container \"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0\": container with ID starting with 27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0 not found: ID does not exist" Jun 03 16:00:57.227064 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.227056 2576 scope.go:117] "RemoveContainer" containerID="52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319" Jun 03 16:00:57.227339 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.227314 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319"} err="failed to get container status \"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319\": rpc error: code = NotFound desc = could not find container \"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319\": container with ID starting with 52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319 not found: ID does not exist" Jun 03 16:00:57.227339 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.227335 2576 scope.go:117] "RemoveContainer" containerID="97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22" Jun 03 16:00:57.227575 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.227549 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22"} err="failed to get container status \"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22\": rpc error: code = NotFound desc = could not find container \"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22\": container with ID starting with 97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22 not found: ID does not exist" Jun 03 16:00:57.227688 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.227575 2576 scope.go:117] "RemoveContainer" containerID="8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b" Jun 03 16:00:57.227878 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.227849 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 03 16:00:57.227973 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.227910 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b"} err="failed to get container status \"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b\": rpc error: code = NotFound desc = could not find container \"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b\": container with ID starting with 8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b not found: ID does not exist" Jun 03 16:00:57.227973 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.227953 2576 scope.go:117] "RemoveContainer" containerID="3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae" Jun 03 16:00:57.228233 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.228209 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae"} err="failed to get container status \"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae\": rpc error: code = NotFound desc = could not find container \"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae\": container with ID starting with 3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae not found: ID does not exist" Jun 03 16:00:57.228309 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.228234 2576 scope.go:117] "RemoveContainer" containerID="9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c" Jun 03 16:00:57.228473 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.228450 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c"} err="failed to get container status \"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c\": rpc error: code = NotFound desc = could not find container \"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c\": container with ID starting with 9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c not found: ID does not exist" Jun 03 16:00:57.228473 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.228471 2576 scope.go:117] "RemoveContainer" containerID="9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df" Jun 03 16:00:57.228754 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.228730 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df"} err="failed to get container status \"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df\": rpc error: code = NotFound desc = could not find container \"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df\": container with ID starting with 9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df not found: ID does not exist" Jun 03 16:00:57.228856 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.228756 2576 scope.go:117] "RemoveContainer" containerID="27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0" Jun 03 16:00:57.229054 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.229029 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0"} err="failed to get container status \"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0\": rpc error: code = NotFound desc = could not find container \"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0\": container with ID starting with 27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0 not found: ID does not exist" Jun 03 16:00:57.229121 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.229057 2576 scope.go:117] "RemoveContainer" containerID="52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319" Jun 03 16:00:57.229295 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.229277 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319"} err="failed to get container status \"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319\": rpc error: code = NotFound desc = could not find container \"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319\": container with ID starting with 52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319 not found: ID does not exist" Jun 03 16:00:57.229366 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.229294 2576 scope.go:117] "RemoveContainer" containerID="97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22" Jun 03 16:00:57.229661 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.229561 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22"} err="failed to get container status \"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22\": rpc error: code = NotFound desc = could not find container \"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22\": container with ID starting with 97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22 not found: ID does not exist" Jun 03 16:00:57.229661 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.229591 2576 scope.go:117] "RemoveContainer" containerID="8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b" Jun 03 16:00:57.230530 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.229711 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 03 16:00:57.230530 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.229879 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b"} err="failed to get container status \"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b\": rpc error: code = NotFound desc = could not find container \"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b\": container with ID starting with 8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b not found: ID does not exist" Jun 03 16:00:57.230530 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.229902 2576 scope.go:117] "RemoveContainer" containerID="3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae" Jun 03 16:00:57.230530 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.230197 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae"} err="failed to get container status \"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae\": rpc error: code = NotFound desc = could not find container \"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae\": container with ID starting with 3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae not found: ID does not exist" Jun 03 16:00:57.230530 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.230234 2576 scope.go:117] "RemoveContainer" containerID="9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c" Jun 03 16:00:57.230530 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.230477 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c"} err="failed to get container status \"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c\": rpc error: code = NotFound desc = could not find container \"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c\": container with ID starting with 9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c not found: ID does not exist" Jun 03 16:00:57.230530 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.230510 2576 scope.go:117] "RemoveContainer" containerID="9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df" Jun 03 16:00:57.231267 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.231036 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df"} err="failed to get container status \"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df\": rpc error: code = NotFound desc = could not find container \"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df\": container with ID starting with 9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df not found: ID does not exist" Jun 03 16:00:57.231267 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.231066 2576 scope.go:117] "RemoveContainer" containerID="27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0" Jun 03 16:00:57.232347 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.232321 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0"} err="failed to get container status \"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0\": rpc error: code = NotFound desc = could not find container \"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0\": container with ID starting with 27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0 not found: ID does not exist" Jun 03 16:00:57.232453 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.232349 2576 scope.go:117] "RemoveContainer" containerID="52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319" Jun 03 16:00:57.232542 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.232521 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 16:00:57.232648 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.232590 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319"} err="failed to get container status \"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319\": rpc error: code = NotFound desc = could not find container \"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319\": container with ID starting with 52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319 not found: ID does not exist" Jun 03 16:00:57.232722 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.232650 2576 scope.go:117] "RemoveContainer" containerID="97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22" Jun 03 16:00:57.232949 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.232927 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22"} err="failed to get container status \"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22\": rpc error: code = NotFound desc = could not find container \"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22\": container with ID starting with 97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22 not found: ID does not exist" Jun 03 16:00:57.232949 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.232948 2576 scope.go:117] "RemoveContainer" containerID="8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b" Jun 03 16:00:57.233182 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.233162 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b"} err="failed to get container status \"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b\": rpc error: code = NotFound desc = could not find container \"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b\": container with ID starting with 8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b not found: ID does not exist" Jun 03 16:00:57.233256 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.233184 2576 scope.go:117] "RemoveContainer" containerID="3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae" Jun 03 16:00:57.233434 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.233413 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae"} err="failed to get container status \"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae\": rpc error: code = NotFound desc = could not find container \"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae\": container with ID starting with 3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae not found: ID does not exist" Jun 03 16:00:57.233434 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.233433 2576 scope.go:117] "RemoveContainer" containerID="9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c" Jun 03 16:00:57.233658 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.233642 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c"} err="failed to get container status \"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c\": rpc error: code = NotFound desc = could not find container \"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c\": container with ID starting with 9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c not found: ID does not exist" Jun 03 16:00:57.233705 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.233658 2576 scope.go:117] "RemoveContainer" containerID="9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df" Jun 03 16:00:57.233911 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.233896 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df"} err="failed to get container status \"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df\": rpc error: code = NotFound desc = could not find container \"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df\": container with ID starting with 9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df not found: ID does not exist" Jun 03 16:00:57.233973 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.233912 2576 scope.go:117] "RemoveContainer" containerID="27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0" Jun 03 16:00:57.234120 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.234098 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0"} err="failed to get container status \"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0\": rpc error: code = NotFound desc = could not find container \"27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0\": container with ID starting with 27c2eb0300482c70bde2c66e4e79009b3959e0f58f656cb30480a38ef23cece0 not found: ID does not exist" Jun 03 16:00:57.234166 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.234122 2576 scope.go:117] "RemoveContainer" containerID="52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319" Jun 03 16:00:57.234398 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.234373 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319"} err="failed to get container status \"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319\": rpc error: code = NotFound desc = could not find container \"52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319\": container with ID starting with 52e36de87fa8a588bfa45b1d440df4c92819885cac33aa68484a1c479508a319 not found: ID does not exist" Jun 03 16:00:57.234474 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.234401 2576 scope.go:117] "RemoveContainer" containerID="97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22" Jun 03 16:00:57.234681 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.234660 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22"} err="failed to get container status \"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22\": rpc error: code = NotFound desc = could not find container \"97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22\": container with ID starting with 97076d8f1c435f91d1b2262fcd1fdce1a61910b6d9259b89fd3b5f0e73368e22 not found: ID does not exist" Jun 03 16:00:57.234681 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.234680 2576 scope.go:117] "RemoveContainer" containerID="8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b" Jun 03 16:00:57.234961 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.234938 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b"} err="failed to get container status \"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b\": rpc error: code = NotFound desc = could not find container \"8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b\": container with ID starting with 8a147c8be9b3579e2219d404f5bca84a1fc42e40a6b0fcd7f3b34e774ddca34b not found: ID does not exist" Jun 03 16:00:57.234961 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.234962 2576 scope.go:117] "RemoveContainer" containerID="3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae" Jun 03 16:00:57.235185 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.235165 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae"} err="failed to get container status \"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae\": rpc error: code = NotFound desc = could not find container \"3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae\": container with ID starting with 3c93bd1ccf748f02e07863a443c6c7f0c044d8dc4aeec5a42ee782aaad7792ae not found: ID does not exist" Jun 03 16:00:57.235226 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.235185 2576 scope.go:117] "RemoveContainer" containerID="9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c" Jun 03 16:00:57.235416 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.235398 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c"} err="failed to get container status \"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c\": rpc error: code = NotFound desc = could not find container \"9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c\": container with ID starting with 9aaca1efd0c223224ab4b55a603b9937f7f6f300106f3a533545d83b72ec770c not found: ID does not exist" Jun 03 16:00:57.235481 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.235418 2576 scope.go:117] "RemoveContainer" containerID="9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df" Jun 03 16:00:57.235651 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.235629 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df"} err="failed to get container status \"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df\": rpc error: code = NotFound desc = could not find container \"9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df\": container with ID starting with 9f1937073a835c1d72ddb1940604a8472e3a2ff03636f1eb68dd0b64fd0e60df not found: ID does not exist" Jun 03 16:00:57.257094 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257058 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/51a11107-fb97-4a27-9810-59b4616cd711-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257246 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257101 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257246 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257120 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51a11107-fb97-4a27-9810-59b4616cd711-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257246 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257152 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257246 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257191 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257246 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257212 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rvqt\" (UniqueName: \"kubernetes.io/projected/51a11107-fb97-4a27-9810-59b4616cd711-kube-api-access-8rvqt\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257246 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257236 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257487 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257251 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/51a11107-fb97-4a27-9810-59b4616cd711-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257487 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257274 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-web-config\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257487 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257305 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51a11107-fb97-4a27-9810-59b4616cd711-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257487 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257333 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257487 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257375 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51a11107-fb97-4a27-9810-59b4616cd711-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257487 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257393 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257487 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257410 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51a11107-fb97-4a27-9810-59b4616cd711-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257487 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257425 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51a11107-fb97-4a27-9810-59b4616cd711-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257487 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257444 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257839 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257505 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51a11107-fb97-4a27-9810-59b4616cd711-config-out\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.257839 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.257587 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-config\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.357975 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.357936 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51a11107-fb97-4a27-9810-59b4616cd711-config-out\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.357975 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.357982 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-config\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.358228 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358004 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/51a11107-fb97-4a27-9810-59b4616cd711-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.358228 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358032 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.358228 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358114 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51a11107-fb97-4a27-9810-59b4616cd711-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.358228 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358150 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.358228 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358182 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.358228 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358208 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8rvqt\" (UniqueName: \"kubernetes.io/projected/51a11107-fb97-4a27-9810-59b4616cd711-kube-api-access-8rvqt\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.362318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358233 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.362318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358257 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/51a11107-fb97-4a27-9810-59b4616cd711-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.362318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358294 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-web-config\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.362318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358330 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51a11107-fb97-4a27-9810-59b4616cd711-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.362318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358355 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.362318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358402 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51a11107-fb97-4a27-9810-59b4616cd711-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.362318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358433 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.362318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358439 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/51a11107-fb97-4a27-9810-59b4616cd711-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.362318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358894 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51a11107-fb97-4a27-9810-59b4616cd711-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.362318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358956 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51a11107-fb97-4a27-9810-59b4616cd711-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.362318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.358987 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51a11107-fb97-4a27-9810-59b4616cd711-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.362318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.359937 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51a11107-fb97-4a27-9810-59b4616cd711-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.362318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.360003 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51a11107-fb97-4a27-9810-59b4616cd711-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.362318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.360676 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51a11107-fb97-4a27-9810-59b4616cd711-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.362318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.360747 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.362318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.362285 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.363151 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.362425 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51a11107-fb97-4a27-9810-59b4616cd711-config-out\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.363151 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.362708 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-web-config\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.363151 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.362924 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.363296 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.363183 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/51a11107-fb97-4a27-9810-59b4616cd711-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.363501 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.363446 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51a11107-fb97-4a27-9810-59b4616cd711-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.363501 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.363464 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.363656 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.363588 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.363721 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.363660 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.363829 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.363812 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-config\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.364024 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.364008 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.364609 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.364593 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/51a11107-fb97-4a27-9810-59b4616cd711-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.366683 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.366665 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rvqt\" (UniqueName: \"kubernetes.io/projected/51a11107-fb97-4a27-9810-59b4616cd711-kube-api-access-8rvqt\") pod \"prometheus-k8s-0\" (UID: \"51a11107-fb97-4a27-9810-59b4616cd711\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.390098 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.390067 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae32d7e7-853b-4459-b854-aa33aeacc8f9" path="/var/lib/kubelet/pods/ae32d7e7-853b-4459-b854-aa33aeacc8f9/volumes" Jun 03 16:00:57.535804 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.535743 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:00:57.666719 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:57.666684 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 03 16:00:57.669655 ip-10-0-128-61 kubenswrapper[2576]: W0603 16:00:57.669628 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51a11107_fb97_4a27_9810_59b4616cd711.slice/crio-956e43a48848182f793a7e10a92fcb20eca2e9c99a39c45c0538ff0989ef6db5 WatchSource:0}: Error finding container 956e43a48848182f793a7e10a92fcb20eca2e9c99a39c45c0538ff0989ef6db5: Status 404 returned error can't find the container with id 956e43a48848182f793a7e10a92fcb20eca2e9c99a39c45c0538ff0989ef6db5 Jun 03 16:00:58.162700 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:58.162670 2576 generic.go:358] "Generic (PLEG): container finished" podID="51a11107-fb97-4a27-9810-59b4616cd711" containerID="bc734e0fb8211ee8c4ce73b910617c4a5f8881e01e26ae89fee6a291a5dc6f44" exitCode=0 Jun 03 16:00:58.162912 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:58.162756 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51a11107-fb97-4a27-9810-59b4616cd711","Type":"ContainerDied","Data":"bc734e0fb8211ee8c4ce73b910617c4a5f8881e01e26ae89fee6a291a5dc6f44"} Jun 03 16:00:58.162912 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:58.162818 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51a11107-fb97-4a27-9810-59b4616cd711","Type":"ContainerStarted","Data":"956e43a48848182f793a7e10a92fcb20eca2e9c99a39c45c0538ff0989ef6db5"} Jun 03 16:00:59.170520 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:59.170482 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51a11107-fb97-4a27-9810-59b4616cd711","Type":"ContainerStarted","Data":"77b379ab427859947ef27fc90b76817083f74bf4cdb0c0099bc17558d149e33d"} Jun 03 16:00:59.170520 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:59.170523 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51a11107-fb97-4a27-9810-59b4616cd711","Type":"ContainerStarted","Data":"7b179141825b07e9b9b9c6559707dd79c53abeb6d6fc1191e300d70ca86ddfa2"} Jun 03 16:00:59.170970 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:59.170535 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51a11107-fb97-4a27-9810-59b4616cd711","Type":"ContainerStarted","Data":"f2bbb5b38f1df284cfff7d0c3b1ba79a7d4e6c0847cc9355fe592dc150f82a4e"} Jun 03 16:00:59.170970 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:59.170544 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51a11107-fb97-4a27-9810-59b4616cd711","Type":"ContainerStarted","Data":"f5b7041323465766af5ba01510b8c98ddb96de82050d76a0df467e593074c33b"} Jun 03 16:00:59.170970 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:59.170557 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51a11107-fb97-4a27-9810-59b4616cd711","Type":"ContainerStarted","Data":"bbe6eb325ea4c5547f1426859b3f5dd12fba99f9a2ee928fa7d31630df2aecf3"} Jun 03 16:00:59.170970 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:59.170566 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51a11107-fb97-4a27-9810-59b4616cd711","Type":"ContainerStarted","Data":"18d2b76c8570738e54c7db37b53024a9d0f25a94736d9b2bf739be99eb68a55d"} Jun 03 16:00:59.198361 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:00:59.198308 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.198292796 podStartE2EDuration="2.198292796s" podCreationTimestamp="2026-06-03 16:00:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:00:59.195524395 +0000 UTC m=+232.372494205" watchObservedRunningTime="2026-06-03 16:00:59.198292796 +0000 UTC m=+232.375262604" Jun 03 16:01:02.536145 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:01:02.536093 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:01:57.536928 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:01:57.536889 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:01:57.552487 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:01:57.552458 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:01:58.359128 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:01:58.359096 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 03 16:02:07.271015 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:02:07.270984 2576 kubelet.go:1628] "Image garbage collection succeeded" Jun 03 16:05:53.612584 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:05:53.612509 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq"] Jun 03 16:05:53.615526 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:05:53.615504 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq" Jun 03 16:05:53.617812 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:05:53.617787 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-kdr7l\"/\"openshift-service-ca.crt\"" Jun 03 16:05:53.618661 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:05:53.618643 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-kdr7l\"/\"kube-root-ca.crt\"" Jun 03 16:05:53.618661 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:05:53.618649 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-kdr7l\"/\"default-dockercfg-cph5m\"" Jun 03 16:05:53.625742 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:05:53.625702 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq"] Jun 03 16:05:53.785041 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:05:53.784998 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkgjn\" (UniqueName: \"kubernetes.io/projected/f34fa2f2-b522-4c97-b7c2-90922760b56b-kube-api-access-mkgjn\") pod \"test-trainjob-5w24v-node-0-0-fnlwq\" (UID: \"f34fa2f2-b522-4c97-b7c2-90922760b56b\") " pod="test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq" Jun 03 16:05:53.885671 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:05:53.885550 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mkgjn\" (UniqueName: \"kubernetes.io/projected/f34fa2f2-b522-4c97-b7c2-90922760b56b-kube-api-access-mkgjn\") pod \"test-trainjob-5w24v-node-0-0-fnlwq\" (UID: \"f34fa2f2-b522-4c97-b7c2-90922760b56b\") " pod="test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq" Jun 03 16:05:53.894599 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:05:53.894569 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkgjn\" (UniqueName: \"kubernetes.io/projected/f34fa2f2-b522-4c97-b7c2-90922760b56b-kube-api-access-mkgjn\") pod \"test-trainjob-5w24v-node-0-0-fnlwq\" (UID: \"f34fa2f2-b522-4c97-b7c2-90922760b56b\") " pod="test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq" Jun 03 16:05:53.927520 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:05:53.927482 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq" Jun 03 16:05:54.054574 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:05:54.054545 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq"] Jun 03 16:05:54.056868 ip-10-0-128-61 kubenswrapper[2576]: W0603 16:05:54.056841 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf34fa2f2_b522_4c97_b7c2_90922760b56b.slice/crio-e2984278cbf435ac83f8bd32a0fa9d4d22a4c0e364d311c84a98da0f6cdd0904 WatchSource:0}: Error finding container e2984278cbf435ac83f8bd32a0fa9d4d22a4c0e364d311c84a98da0f6cdd0904: Status 404 returned error can't find the container with id e2984278cbf435ac83f8bd32a0fa9d4d22a4c0e364d311c84a98da0f6cdd0904 Jun 03 16:05:54.058747 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:05:54.058729 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:05:55.008944 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:05:55.008898 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq" event={"ID":"f34fa2f2-b522-4c97-b7c2-90922760b56b","Type":"ContainerStarted","Data":"e2984278cbf435ac83f8bd32a0fa9d4d22a4c0e364d311c84a98da0f6cdd0904"} Jun 03 16:11:01.962628 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:01.962595 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq" event={"ID":"f34fa2f2-b522-4c97-b7c2-90922760b56b","Type":"ContainerStarted","Data":"bf3eebe9c811dcadbfac680cc2467efbb0b42b89a1b93368838d7ce3c42d1a95"} Jun 03 16:11:01.984866 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:01.984804 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq" podStartSLOduration=1.915158766 podStartE2EDuration="5m8.984789178s" podCreationTimestamp="2026-06-03 16:05:53 +0000 UTC" firstStartedPulling="2026-06-03 16:05:54.058878015 +0000 UTC m=+527.235847802" lastFinishedPulling="2026-06-03 16:11:01.128508424 +0000 UTC m=+834.305478214" observedRunningTime="2026-06-03 16:11:01.98344299 +0000 UTC m=+835.160412795" watchObservedRunningTime="2026-06-03 16:11:01.984789178 +0000 UTC m=+835.161758981" Jun 03 16:11:02.476318 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:02.476289 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-kdr7l_test-trainjob-5w24v-node-0-0-fnlwq_f34fa2f2-b522-4c97-b7c2-90922760b56b/node/0.log" Jun 03 16:11:05.476726 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:05.476697 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-kdr7l_test-trainjob-5w24v-node-0-0-fnlwq_f34fa2f2-b522-4c97-b7c2-90922760b56b/node/0.log" Jun 03 16:11:08.476508 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:08.476479 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-kdr7l_test-trainjob-5w24v-node-0-0-fnlwq_f34fa2f2-b522-4c97-b7c2-90922760b56b/node/0.log" Jun 03 16:11:08.985028 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:08.984997 2576 generic.go:358] "Generic (PLEG): container finished" podID="f34fa2f2-b522-4c97-b7c2-90922760b56b" containerID="bf3eebe9c811dcadbfac680cc2467efbb0b42b89a1b93368838d7ce3c42d1a95" exitCode=0 Jun 03 16:11:08.985199 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:08.985066 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq" event={"ID":"f34fa2f2-b522-4c97-b7c2-90922760b56b","Type":"ContainerDied","Data":"bf3eebe9c811dcadbfac680cc2467efbb0b42b89a1b93368838d7ce3c42d1a95"} Jun 03 16:11:10.161610 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:10.161584 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq" Jun 03 16:11:10.269102 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:10.269011 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkgjn\" (UniqueName: \"kubernetes.io/projected/f34fa2f2-b522-4c97-b7c2-90922760b56b-kube-api-access-mkgjn\") pod \"f34fa2f2-b522-4c97-b7c2-90922760b56b\" (UID: \"f34fa2f2-b522-4c97-b7c2-90922760b56b\") " Jun 03 16:11:10.271185 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:10.271150 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f34fa2f2-b522-4c97-b7c2-90922760b56b-kube-api-access-mkgjn" (OuterVolumeSpecName: "kube-api-access-mkgjn") pod "f34fa2f2-b522-4c97-b7c2-90922760b56b" (UID: "f34fa2f2-b522-4c97-b7c2-90922760b56b"). InnerVolumeSpecName "kube-api-access-mkgjn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:11:10.370229 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:10.370191 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkgjn\" (UniqueName: \"kubernetes.io/projected/f34fa2f2-b522-4c97-b7c2-90922760b56b-kube-api-access-mkgjn\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:11:10.992949 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:10.992916 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq" Jun 03 16:11:10.992949 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:10.992926 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq" event={"ID":"f34fa2f2-b522-4c97-b7c2-90922760b56b","Type":"ContainerDied","Data":"e2984278cbf435ac83f8bd32a0fa9d4d22a4c0e364d311c84a98da0f6cdd0904"} Jun 03 16:11:10.992949 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:10.992955 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2984278cbf435ac83f8bd32a0fa9d4d22a4c0e364d311c84a98da0f6cdd0904" Jun 03 16:11:11.477203 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:11.477175 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-kdr7l_test-trainjob-5w24v-node-0-0-fnlwq_f34fa2f2-b522-4c97-b7c2-90922760b56b/node/0.log" Jun 03 16:11:11.976062 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:11.976024 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-kdr7l_test-trainjob-5w24v-node-0-0-fnlwq_f34fa2f2-b522-4c97-b7c2-90922760b56b/node/0.log" Jun 03 16:11:13.413381 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.413345 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nlqct/must-gather-r5qf6"] Jun 03 16:11:13.413760 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.413643 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f34fa2f2-b522-4c97-b7c2-90922760b56b" containerName="node" Jun 03 16:11:13.413760 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.413654 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f34fa2f2-b522-4c97-b7c2-90922760b56b" containerName="node" Jun 03 16:11:13.413760 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.413711 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="f34fa2f2-b522-4c97-b7c2-90922760b56b" containerName="node" Jun 03 16:11:13.462786 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.462740 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nlqct/must-gather-r5qf6"] Jun 03 16:11:13.462961 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.462889 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nlqct/must-gather-r5qf6" Jun 03 16:11:13.465508 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.465483 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nlqct\"/\"openshift-service-ca.crt\"" Jun 03 16:11:13.466494 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.466468 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nlqct\"/\"kube-root-ca.crt\"" Jun 03 16:11:13.466621 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.466499 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nlqct\"/\"default-dockercfg-qz96f\"" Jun 03 16:11:13.599410 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.599371 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfb4q\" (UniqueName: \"kubernetes.io/projected/5f09a403-ae9b-4472-9b40-8e846c5bda42-kube-api-access-hfb4q\") pod \"must-gather-r5qf6\" (UID: \"5f09a403-ae9b-4472-9b40-8e846c5bda42\") " pod="openshift-must-gather-nlqct/must-gather-r5qf6" Jun 03 16:11:13.599593 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.599455 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5f09a403-ae9b-4472-9b40-8e846c5bda42-must-gather-output\") pod \"must-gather-r5qf6\" (UID: \"5f09a403-ae9b-4472-9b40-8e846c5bda42\") " pod="openshift-must-gather-nlqct/must-gather-r5qf6" Jun 03 16:11:13.700456 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.700413 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hfb4q\" (UniqueName: \"kubernetes.io/projected/5f09a403-ae9b-4472-9b40-8e846c5bda42-kube-api-access-hfb4q\") pod \"must-gather-r5qf6\" (UID: \"5f09a403-ae9b-4472-9b40-8e846c5bda42\") " pod="openshift-must-gather-nlqct/must-gather-r5qf6" Jun 03 16:11:13.700674 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.700475 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5f09a403-ae9b-4472-9b40-8e846c5bda42-must-gather-output\") pod \"must-gather-r5qf6\" (UID: \"5f09a403-ae9b-4472-9b40-8e846c5bda42\") " pod="openshift-must-gather-nlqct/must-gather-r5qf6" Jun 03 16:11:13.700863 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.700840 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5f09a403-ae9b-4472-9b40-8e846c5bda42-must-gather-output\") pod \"must-gather-r5qf6\" (UID: \"5f09a403-ae9b-4472-9b40-8e846c5bda42\") " pod="openshift-must-gather-nlqct/must-gather-r5qf6" Jun 03 16:11:13.708491 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.708454 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfb4q\" (UniqueName: \"kubernetes.io/projected/5f09a403-ae9b-4472-9b40-8e846c5bda42-kube-api-access-hfb4q\") pod \"must-gather-r5qf6\" (UID: \"5f09a403-ae9b-4472-9b40-8e846c5bda42\") " pod="openshift-must-gather-nlqct/must-gather-r5qf6" Jun 03 16:11:13.772798 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.772727 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nlqct/must-gather-r5qf6" Jun 03 16:11:13.901284 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.901245 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nlqct/must-gather-r5qf6"] Jun 03 16:11:13.904461 ip-10-0-128-61 kubenswrapper[2576]: W0603 16:11:13.904428 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f09a403_ae9b_4472_9b40_8e846c5bda42.slice/crio-e0cb6361edafa878222e5a71507b45852f6176dfd8f2fa765170b7b42c6512aa WatchSource:0}: Error finding container e0cb6361edafa878222e5a71507b45852f6176dfd8f2fa765170b7b42c6512aa: Status 404 returned error can't find the container with id e0cb6361edafa878222e5a71507b45852f6176dfd8f2fa765170b7b42c6512aa Jun 03 16:11:13.906273 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:13.906257 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:11:14.001651 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:14.001567 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nlqct/must-gather-r5qf6" event={"ID":"5f09a403-ae9b-4472-9b40-8e846c5bda42","Type":"ContainerStarted","Data":"e0cb6361edafa878222e5a71507b45852f6176dfd8f2fa765170b7b42c6512aa"} Jun 03 16:11:17.004911 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:17.004872 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq"] Jun 03 16:11:17.010094 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:17.010061 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-kdr7l/test-trainjob-5w24v-node-0-0-fnlwq"] Jun 03 16:11:17.390545 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:17.390466 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f34fa2f2-b522-4c97-b7c2-90922760b56b" path="/var/lib/kubelet/pods/f34fa2f2-b522-4c97-b7c2-90922760b56b/volumes" Jun 03 16:11:22.031673 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:22.031638 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nlqct/must-gather-r5qf6" event={"ID":"5f09a403-ae9b-4472-9b40-8e846c5bda42","Type":"ContainerStarted","Data":"158389de5114869a2790d28c8c0d5899e84c1ffbe227ea8db1d7ca3124d126b2"} Jun 03 16:11:22.031673 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:22.031675 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nlqct/must-gather-r5qf6" event={"ID":"5f09a403-ae9b-4472-9b40-8e846c5bda42","Type":"ContainerStarted","Data":"12c6f7889ef9fdcf42479652e794677dad5b26af35d9a4f17b483dfb53b760e4"} Jun 03 16:11:22.046917 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:11:22.046855 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-nlqct/must-gather-r5qf6" podStartSLOduration=2.058285932 podStartE2EDuration="9.046838326s" podCreationTimestamp="2026-06-03 16:11:13 +0000 UTC" firstStartedPulling="2026-06-03 16:11:13.906381497 +0000 UTC m=+847.083351284" lastFinishedPulling="2026-06-03 16:11:20.894933883 +0000 UTC m=+854.071903678" observedRunningTime="2026-06-03 16:11:22.046010444 +0000 UTC m=+855.222980248" watchObservedRunningTime="2026-06-03 16:11:22.046838326 +0000 UTC m=+855.223808168" Jun 03 16:12:12.195154 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:12.195111 2576 generic.go:358] "Generic (PLEG): container finished" podID="5f09a403-ae9b-4472-9b40-8e846c5bda42" containerID="12c6f7889ef9fdcf42479652e794677dad5b26af35d9a4f17b483dfb53b760e4" exitCode=0 Jun 03 16:12:12.195673 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:12.195171 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nlqct/must-gather-r5qf6" event={"ID":"5f09a403-ae9b-4472-9b40-8e846c5bda42","Type":"ContainerDied","Data":"12c6f7889ef9fdcf42479652e794677dad5b26af35d9a4f17b483dfb53b760e4"} Jun 03 16:12:12.195673 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:12.195591 2576 scope.go:117] "RemoveContainer" containerID="12c6f7889ef9fdcf42479652e794677dad5b26af35d9a4f17b483dfb53b760e4" Jun 03 16:12:12.730569 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:12.730538 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-nlqct_must-gather-r5qf6_5f09a403-ae9b-4472-9b40-8e846c5bda42/gather/0.log" Jun 03 16:12:16.035683 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:16.035652 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zfdzl_d8f8466e-baef-4149-964b-a5366d798c52/global-pull-secret-syncer/0.log" Jun 03 16:12:16.118927 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:16.118898 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7w5ch_128d979f-9fbc-4899-8f0a-b0023e619799/konnectivity-agent/0.log" Jun 03 16:12:16.173410 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:16.173367 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-61.ec2.internal_790a10cb9330d251bd135e9345ddb3cd/haproxy/0.log" Jun 03 16:12:18.086393 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:18.086355 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-nlqct/must-gather-r5qf6"] Jun 03 16:12:18.086894 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:18.086580 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-must-gather-nlqct/must-gather-r5qf6" podUID="5f09a403-ae9b-4472-9b40-8e846c5bda42" containerName="copy" containerID="cri-o://158389de5114869a2790d28c8c0d5899e84c1ffbe227ea8db1d7ca3124d126b2" gracePeriod=2 Jun 03 16:12:18.091201 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:18.091174 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-nlqct/must-gather-r5qf6"] Jun 03 16:12:18.214594 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:18.214560 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-nlqct_must-gather-r5qf6_5f09a403-ae9b-4472-9b40-8e846c5bda42/copy/0.log" Jun 03 16:12:18.214986 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:18.214959 2576 generic.go:358] "Generic (PLEG): container finished" podID="5f09a403-ae9b-4472-9b40-8e846c5bda42" containerID="158389de5114869a2790d28c8c0d5899e84c1ffbe227ea8db1d7ca3124d126b2" exitCode=143 Jun 03 16:12:18.317649 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:18.317617 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-nlqct_must-gather-r5qf6_5f09a403-ae9b-4472-9b40-8e846c5bda42/copy/0.log" Jun 03 16:12:18.318008 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:18.317991 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nlqct/must-gather-r5qf6" Jun 03 16:12:18.319982 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:18.319960 2576 status_manager.go:895] "Failed to get status for pod" podUID="5f09a403-ae9b-4472-9b40-8e846c5bda42" pod="openshift-must-gather-nlqct/must-gather-r5qf6" err="pods \"must-gather-r5qf6\" is forbidden: User \"system:node:ip-10-0-128-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-nlqct\": no relationship found between node 'ip-10-0-128-61.ec2.internal' and this object" Jun 03 16:12:18.462863 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:18.462824 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5f09a403-ae9b-4472-9b40-8e846c5bda42-must-gather-output\") pod \"5f09a403-ae9b-4472-9b40-8e846c5bda42\" (UID: \"5f09a403-ae9b-4472-9b40-8e846c5bda42\") " Jun 03 16:12:18.463034 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:18.462919 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfb4q\" (UniqueName: \"kubernetes.io/projected/5f09a403-ae9b-4472-9b40-8e846c5bda42-kube-api-access-hfb4q\") pod \"5f09a403-ae9b-4472-9b40-8e846c5bda42\" (UID: \"5f09a403-ae9b-4472-9b40-8e846c5bda42\") " Jun 03 16:12:18.464998 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:18.464966 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f09a403-ae9b-4472-9b40-8e846c5bda42-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "5f09a403-ae9b-4472-9b40-8e846c5bda42" (UID: "5f09a403-ae9b-4472-9b40-8e846c5bda42"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:12:18.471551 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:18.471515 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f09a403-ae9b-4472-9b40-8e846c5bda42-kube-api-access-hfb4q" (OuterVolumeSpecName: "kube-api-access-hfb4q") pod "5f09a403-ae9b-4472-9b40-8e846c5bda42" (UID: "5f09a403-ae9b-4472-9b40-8e846c5bda42"). InnerVolumeSpecName "kube-api-access-hfb4q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:12:18.563701 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:18.563658 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfb4q\" (UniqueName: \"kubernetes.io/projected/5f09a403-ae9b-4472-9b40-8e846c5bda42-kube-api-access-hfb4q\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:12:18.563701 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:18.563700 2576 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5f09a403-ae9b-4472-9b40-8e846c5bda42-must-gather-output\") on node \"ip-10-0-128-61.ec2.internal\" DevicePath \"\"" Jun 03 16:12:19.219536 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.219510 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-nlqct_must-gather-r5qf6_5f09a403-ae9b-4472-9b40-8e846c5bda42/copy/0.log" Jun 03 16:12:19.219958 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.219905 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nlqct/must-gather-r5qf6" Jun 03 16:12:19.219958 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.219937 2576 scope.go:117] "RemoveContainer" containerID="158389de5114869a2790d28c8c0d5899e84c1ffbe227ea8db1d7ca3124d126b2" Jun 03 16:12:19.222354 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.222322 2576 status_manager.go:895] "Failed to get status for pod" podUID="5f09a403-ae9b-4472-9b40-8e846c5bda42" pod="openshift-must-gather-nlqct/must-gather-r5qf6" err="pods \"must-gather-r5qf6\" is forbidden: User \"system:node:ip-10-0-128-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-nlqct\": no relationship found between node 'ip-10-0-128-61.ec2.internal' and this object" Jun 03 16:12:19.227615 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.227278 2576 scope.go:117] "RemoveContainer" containerID="12c6f7889ef9fdcf42479652e794677dad5b26af35d9a4f17b483dfb53b760e4" Jun 03 16:12:19.229938 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.229902 2576 status_manager.go:895] "Failed to get status for pod" podUID="5f09a403-ae9b-4472-9b40-8e846c5bda42" pod="openshift-must-gather-nlqct/must-gather-r5qf6" err="pods \"must-gather-r5qf6\" is forbidden: User \"system:node:ip-10-0-128-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-nlqct\": no relationship found between node 'ip-10-0-128-61.ec2.internal' and this object" Jun 03 16:12:19.389210 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.389129 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f09a403-ae9b-4472-9b40-8e846c5bda42" path="/var/lib/kubelet/pods/5f09a403-ae9b-4472-9b40-8e846c5bda42/volumes" Jun 03 16:12:19.410544 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.410515 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-66l9q_1b481d2d-e16a-4f9c-a022-cbcc232ced7b/node-exporter/0.log" Jun 03 16:12:19.430575 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.430549 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-66l9q_1b481d2d-e16a-4f9c-a022-cbcc232ced7b/kube-rbac-proxy/0.log" Jun 03 16:12:19.455803 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.455763 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-66l9q_1b481d2d-e16a-4f9c-a022-cbcc232ced7b/init-textfile/0.log" Jun 03 16:12:19.731804 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.731754 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51a11107-fb97-4a27-9810-59b4616cd711/prometheus/0.log" Jun 03 16:12:19.751389 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.751361 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51a11107-fb97-4a27-9810-59b4616cd711/config-reloader/0.log" Jun 03 16:12:19.772322 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.772297 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51a11107-fb97-4a27-9810-59b4616cd711/thanos-sidecar/0.log" Jun 03 16:12:19.791597 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.791570 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51a11107-fb97-4a27-9810-59b4616cd711/kube-rbac-proxy-web/0.log" Jun 03 16:12:19.811309 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.811284 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51a11107-fb97-4a27-9810-59b4616cd711/kube-rbac-proxy/0.log" Jun 03 16:12:19.835931 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.835897 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51a11107-fb97-4a27-9810-59b4616cd711/kube-rbac-proxy-thanos/0.log" Jun 03 16:12:19.857021 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.856993 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51a11107-fb97-4a27-9810-59b4616cd711/init-config-reloader/0.log" Jun 03 16:12:19.925657 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.925625 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-59b7f8756b-nchxs_86fac976-acfe-4419-9e2a-0f468bed1f6d/prometheus-operator-admission-webhook/0.log" Jun 03 16:12:19.954144 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.954102 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-555766866-dzx92_d37840e1-e034-43a9-907d-29543d05e389/telemeter-client/0.log" Jun 03 16:12:19.975512 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.975484 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-555766866-dzx92_d37840e1-e034-43a9-907d-29543d05e389/reload/0.log" Jun 03 16:12:19.994903 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:19.994818 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-555766866-dzx92_d37840e1-e034-43a9-907d-29543d05e389/kube-rbac-proxy/0.log" Jun 03 16:12:21.263566 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:21.263532 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-76c9bb5fdf-k7xdg_4f6ff88f-c358-44b6-aa8e-ee0b2f094036/networking-console-plugin/0.log" Jun 03 16:12:22.397151 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:22.397122 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-7598fd5bd-9npcq_d0510938-d2f4-4c49-a674-b893977e8443/volume-data-source-validator/0.log" Jun 03 16:12:22.870271 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:22.870226 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm"] Jun 03 16:12:22.870541 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:22.870527 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f09a403-ae9b-4472-9b40-8e846c5bda42" containerName="gather" Jun 03 16:12:22.870541 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:22.870542 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f09a403-ae9b-4472-9b40-8e846c5bda42" containerName="gather" Jun 03 16:12:22.870635 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:22.870557 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f09a403-ae9b-4472-9b40-8e846c5bda42" containerName="copy" Jun 03 16:12:22.870635 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:22.870563 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f09a403-ae9b-4472-9b40-8e846c5bda42" containerName="copy" Jun 03 16:12:22.870695 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:22.870636 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="5f09a403-ae9b-4472-9b40-8e846c5bda42" containerName="copy" Jun 03 16:12:22.870695 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:22.870648 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="5f09a403-ae9b-4472-9b40-8e846c5bda42" containerName="gather" Jun 03 16:12:22.872482 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:22.872464 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:22.875151 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:22.875126 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mnc5j\"/\"default-dockercfg-wrfwr\"" Jun 03 16:12:22.875268 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:22.875208 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mnc5j\"/\"kube-root-ca.crt\"" Jun 03 16:12:22.875986 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:22.875969 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mnc5j\"/\"openshift-service-ca.crt\"" Jun 03 16:12:22.883896 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:22.883871 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm"] Jun 03 16:12:23.002089 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.002044 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ccfde265-b188-4be6-b300-a7e59bed676f-proc\") pod \"perf-node-gather-daemonset-f9jcm\" (UID: \"ccfde265-b188-4be6-b300-a7e59bed676f\") " pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:23.002089 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.002087 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ccfde265-b188-4be6-b300-a7e59bed676f-lib-modules\") pod \"perf-node-gather-daemonset-f9jcm\" (UID: \"ccfde265-b188-4be6-b300-a7e59bed676f\") " pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:23.002292 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.002179 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccfde265-b188-4be6-b300-a7e59bed676f-sys\") pod \"perf-node-gather-daemonset-f9jcm\" (UID: \"ccfde265-b188-4be6-b300-a7e59bed676f\") " pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:23.002292 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.002215 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ccfde265-b188-4be6-b300-a7e59bed676f-podres\") pod \"perf-node-gather-daemonset-f9jcm\" (UID: \"ccfde265-b188-4be6-b300-a7e59bed676f\") " pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:23.002292 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.002243 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sb6l\" (UniqueName: \"kubernetes.io/projected/ccfde265-b188-4be6-b300-a7e59bed676f-kube-api-access-9sb6l\") pod \"perf-node-gather-daemonset-f9jcm\" (UID: \"ccfde265-b188-4be6-b300-a7e59bed676f\") " pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:23.026351 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.026323 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-76kk6_649647f7-81ce-4ed1-80d4-7ab2cabda43d/dns/0.log" Jun 03 16:12:23.046826 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.046801 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-76kk6_649647f7-81ce-4ed1-80d4-7ab2cabda43d/kube-rbac-proxy/0.log" Jun 03 16:12:23.103048 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.103008 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccfde265-b188-4be6-b300-a7e59bed676f-sys\") pod \"perf-node-gather-daemonset-f9jcm\" (UID: \"ccfde265-b188-4be6-b300-a7e59bed676f\") " pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:23.103048 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.103048 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ccfde265-b188-4be6-b300-a7e59bed676f-podres\") pod \"perf-node-gather-daemonset-f9jcm\" (UID: \"ccfde265-b188-4be6-b300-a7e59bed676f\") " pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:23.103303 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.103073 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9sb6l\" (UniqueName: \"kubernetes.io/projected/ccfde265-b188-4be6-b300-a7e59bed676f-kube-api-access-9sb6l\") pod \"perf-node-gather-daemonset-f9jcm\" (UID: \"ccfde265-b188-4be6-b300-a7e59bed676f\") " pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:23.103303 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.103114 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ccfde265-b188-4be6-b300-a7e59bed676f-proc\") pod \"perf-node-gather-daemonset-f9jcm\" (UID: \"ccfde265-b188-4be6-b300-a7e59bed676f\") " pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:23.103303 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.103132 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ccfde265-b188-4be6-b300-a7e59bed676f-lib-modules\") pod \"perf-node-gather-daemonset-f9jcm\" (UID: \"ccfde265-b188-4be6-b300-a7e59bed676f\") " pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:23.103303 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.103133 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccfde265-b188-4be6-b300-a7e59bed676f-sys\") pod \"perf-node-gather-daemonset-f9jcm\" (UID: \"ccfde265-b188-4be6-b300-a7e59bed676f\") " pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:23.103303 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.103202 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ccfde265-b188-4be6-b300-a7e59bed676f-podres\") pod \"perf-node-gather-daemonset-f9jcm\" (UID: \"ccfde265-b188-4be6-b300-a7e59bed676f\") " pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:23.103303 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.103214 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ccfde265-b188-4be6-b300-a7e59bed676f-proc\") pod \"perf-node-gather-daemonset-f9jcm\" (UID: \"ccfde265-b188-4be6-b300-a7e59bed676f\") " pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:23.103303 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.103274 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ccfde265-b188-4be6-b300-a7e59bed676f-lib-modules\") pod \"perf-node-gather-daemonset-f9jcm\" (UID: \"ccfde265-b188-4be6-b300-a7e59bed676f\") " pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:23.111876 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.111848 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sb6l\" (UniqueName: \"kubernetes.io/projected/ccfde265-b188-4be6-b300-a7e59bed676f-kube-api-access-9sb6l\") pod \"perf-node-gather-daemonset-f9jcm\" (UID: \"ccfde265-b188-4be6-b300-a7e59bed676f\") " pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:23.182110 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.182022 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:23.213137 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.213108 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q5cmz_16c438af-88c9-4b06-be30-c984a697afeb/dns-node-resolver/0.log" Jun 03 16:12:23.312951 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.312919 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm"] Jun 03 16:12:23.316683 ip-10-0-128-61 kubenswrapper[2576]: W0603 16:12:23.316644 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podccfde265_b188_4be6_b300_a7e59bed676f.slice/crio-425b22ac681c40af9aa6218193400163ccd5c849f8639e3bc0c84f3067954000 WatchSource:0}: Error finding container 425b22ac681c40af9aa6218193400163ccd5c849f8639e3bc0c84f3067954000: Status 404 returned error can't find the container with id 425b22ac681c40af9aa6218193400163ccd5c849f8639e3bc0c84f3067954000 Jun 03 16:12:23.613518 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.613491 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-658b55885b-dt9vz_05a59322-0f9f-427a-8cfc-2eeb363397f5/registry/0.log" Jun 03 16:12:23.616323 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.616300 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-658b55885b-dt9vz_05a59322-0f9f-427a-8cfc-2eeb363397f5/registry/1.log" Jun 03 16:12:23.635392 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:23.635359 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4ppxs_0aa350a0-203c-4286-b184-d62efbd0af97/node-ca/0.log" Jun 03 16:12:24.238134 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:24.238093 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" event={"ID":"ccfde265-b188-4be6-b300-a7e59bed676f","Type":"ContainerStarted","Data":"d2f61442cbe9781218a416c7581dbc45cd3e3a481aaafcb256ee351311dadf36"} Jun 03 16:12:24.238134 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:24.238140 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" event={"ID":"ccfde265-b188-4be6-b300-a7e59bed676f","Type":"ContainerStarted","Data":"425b22ac681c40af9aa6218193400163ccd5c849f8639e3bc0c84f3067954000"} Jun 03 16:12:24.238356 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:24.238220 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:24.254310 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:24.254247 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" podStartSLOduration=2.254231781 podStartE2EDuration="2.254231781s" podCreationTimestamp="2026-06-03 16:12:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:12:24.252343089 +0000 UTC m=+917.429312898" watchObservedRunningTime="2026-06-03 16:12:24.254231781 +0000 UTC m=+917.431201590" Jun 03 16:12:24.352810 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:24.352780 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-f7f955c5f-45jt2_3fae3b07-0b61-4b4c-84e3-c6203b79c4af/router/0.log" Jun 03 16:12:24.668690 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:24.668612 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l2jxd_1e3e630c-6304-40da-935c-2dc17eb2ddd0/serve-healthcheck-canary/0.log" Jun 03 16:12:25.096989 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:25.096956 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kfhg4_d894be86-2b42-4354-aebe-c3533846e9dd/kube-rbac-proxy/0.log" Jun 03 16:12:25.117700 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:25.117667 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kfhg4_d894be86-2b42-4354-aebe-c3533846e9dd/exporter/0.log" Jun 03 16:12:25.136507 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:25.136484 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kfhg4_d894be86-2b42-4354-aebe-c3533846e9dd/extractor/0.log" Jun 03 16:12:29.885672 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:29.885643 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-vbrqv_6ec5c06b-c6ab-4826-9076-93046a57fb73/kube-storage-version-migrator-operator/1.log" Jun 03 16:12:29.886899 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:29.886878 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-vbrqv_6ec5c06b-c6ab-4826-9076-93046a57fb73/kube-storage-version-migrator-operator/0.log" Jun 03 16:12:30.250555 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:30.250525 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mnc5j/perf-node-gather-daemonset-f9jcm" Jun 03 16:12:30.808576 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:30.808545 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j7xtj_27386c54-e43c-47d4-89e2-7ae61fb82e65/kube-multus-additional-cni-plugins/0.log" Jun 03 16:12:30.849649 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:30.849620 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j7xtj_27386c54-e43c-47d4-89e2-7ae61fb82e65/egress-router-binary-copy/0.log" Jun 03 16:12:30.897794 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:30.897745 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j7xtj_27386c54-e43c-47d4-89e2-7ae61fb82e65/cni-plugins/0.log" Jun 03 16:12:30.916648 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:30.916622 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j7xtj_27386c54-e43c-47d4-89e2-7ae61fb82e65/bond-cni-plugin/0.log" Jun 03 16:12:30.934672 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:30.934642 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j7xtj_27386c54-e43c-47d4-89e2-7ae61fb82e65/routeoverride-cni/0.log" Jun 03 16:12:30.956120 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:30.956097 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j7xtj_27386c54-e43c-47d4-89e2-7ae61fb82e65/whereabouts-cni-bincopy/0.log" Jun 03 16:12:30.973808 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:30.973781 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j7xtj_27386c54-e43c-47d4-89e2-7ae61fb82e65/whereabouts-cni/0.log" Jun 03 16:12:31.242378 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:31.242350 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tdzqn_31941200-bc35-48cc-9148-b104dbfde42e/kube-multus/0.log" Jun 03 16:12:31.364518 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:31.364485 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w5jsj_2b455f74-b695-4158-8fe3-2fe335bd3e3b/network-metrics-daemon/0.log" Jun 03 16:12:31.394630 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:31.394564 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w5jsj_2b455f74-b695-4158-8fe3-2fe335bd3e3b/kube-rbac-proxy/0.log" Jun 03 16:12:32.269435 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:32.269390 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bjv7n_490f0a30-89ae-4e18-87f0-498a714ff08c/ovn-controller/0.log" Jun 03 16:12:32.298005 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:32.297976 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bjv7n_490f0a30-89ae-4e18-87f0-498a714ff08c/ovn-acl-logging/0.log" Jun 03 16:12:32.321741 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:32.321720 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bjv7n_490f0a30-89ae-4e18-87f0-498a714ff08c/kube-rbac-proxy-node/0.log" Jun 03 16:12:32.347304 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:32.347274 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bjv7n_490f0a30-89ae-4e18-87f0-498a714ff08c/kube-rbac-proxy-ovn-metrics/0.log" Jun 03 16:12:32.366071 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:32.366032 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bjv7n_490f0a30-89ae-4e18-87f0-498a714ff08c/northd/0.log" Jun 03 16:12:32.385764 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:32.385729 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bjv7n_490f0a30-89ae-4e18-87f0-498a714ff08c/nbdb/0.log" Jun 03 16:12:32.408209 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:32.408178 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bjv7n_490f0a30-89ae-4e18-87f0-498a714ff08c/sbdb/0.log" Jun 03 16:12:32.504373 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:32.504349 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bjv7n_490f0a30-89ae-4e18-87f0-498a714ff08c/ovnkube-controller/0.log" Jun 03 16:12:34.168430 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:34.168357 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-56d84b9f44-5ctz2_d4445224-20e9-4308-bdcd-b1255e1fee20/check-endpoints/0.log" Jun 03 16:12:34.210721 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:34.210675 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-g74tp_5bcd65d8-45ad-42e2-8554-16e218c5b465/network-check-target-container/0.log" Jun 03 16:12:35.053992 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:35.053961 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-z289q_c1c44950-8787-4b29-9f1b-c6a191f81431/iptables-alerter/0.log" Jun 03 16:12:35.634640 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:35.634609 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-p4fdl_7e1d606f-ce5b-4871-b678-8d93934e06bf/tuned/0.log" Jun 03 16:12:38.488854 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:38.488824 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca_service-ca-685df94d57-zzxsx_f1114a3a-86c1-4af0-a3f2-b27947d5584a/service-ca-controller/0.log" Jun 03 16:12:38.906036 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:38.905959 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-dm5xr_2544ae4d-5cee-47ff-b8db-2e27aefebf5c/csi-driver/0.log" Jun 03 16:12:38.923699 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:38.923668 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-dm5xr_2544ae4d-5cee-47ff-b8db-2e27aefebf5c/csi-node-driver-registrar/0.log" Jun 03 16:12:38.948133 ip-10-0-128-61 kubenswrapper[2576]: I0603 16:12:38.948105 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-dm5xr_2544ae4d-5cee-47ff-b8db-2e27aefebf5c/csi-liveness-probe/0.log"